2024-04-27T05:13:53.895 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-27T05:13:53.895 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743 --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} --name teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/orig.config.yaml 2024-04-27T05:13:53.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T05:13:54.015 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743 branch: reef description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7675743' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 wait-for-scrub: false ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 rgw: ec-data-pool: false frontend: beast thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 5403a02127f27ec6c3309ef057522ab70cdeb9d2 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 4225 sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 sleep_before_teardown: 0 subset: 0/1 suite: rgw suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 5403a02127f27ec6c3309ef057522ab70cdeb9d2 targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIIIWIdHm4s3w36bsFdI0PrVydeWjs+kcw3nwWuwgFCTc0Jp8ALO2Npj8/i2DmSI1SNzXOX3XylubebHIIPmJ5E= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmHQcWGcmLsqTmo2hP/NZlhARlo3SnPTiQJKMcysv2CPO/7aeC4hJlBMWWpG8L+Wr70LQjwbl+zuLv7JQVIT6A= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_22:40:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-27T05:13:54.015 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa; will attempt to use it 2024-04-27T05:13:54.016 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa/tasks 2024-04-27T05:13:54.016 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-27T05:13:54.017 INFO:teuthology.task.internal:Checking packages... 2024-04-27T05:13:54.041 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7d2137f0a666c92a9411c1af5c33e4968be54da6' 2024-04-27T05:13:54.041 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T05:13:54.041 INFO:teuthology.packaging:ref: None 2024-04-27T05:13:54.041 INFO:teuthology.packaging:tag: None 2024-04-27T05:13:54.041 INFO:teuthology.packaging:branch: reef 2024-04-27T05:13:54.041 INFO:teuthology.packaging:sha1: 7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T05:13:54.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-04-27T05:13:54.368 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1260.g5403a021 2024-04-27T05:13:54.369 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-27T05:13:54.376 INFO:teuthology.task.internal:no buildpackages task found 2024-04-27T05:13:54.376 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-27T05:13:54.391 INFO:teuthology.task.internal:Saving configuration 2024-04-27T05:13:54.403 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-27T05:13:54.409 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T05:13:54.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 05:08:55.473802', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIIIWIdHm4s3w36bsFdI0PrVydeWjs+kcw3nwWuwgFCTc0Jp8ALO2Npj8/i2DmSI1SNzXOX3XylubebHIIPmJ5E='} 2024-04-27T05:13:54.452 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 05:08:55.475087', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmHQcWGcmLsqTmo2hP/NZlhARlo3SnPTiQJKMcysv2CPO/7aeC4hJlBMWWpG8L+Wr70LQjwbl+zuLv7JQVIT6A='} 2024-04-27T05:13:54.452 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-27T05:13:54.458 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-04-27T05:13:54.458 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-04-27T05:13:54.458 INFO:teuthology.run_tasks:Running task console_log... 2024-04-27T05:13:54.533 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f77cfa1a3a0>, signals=[15]) 2024-04-27T05:13:54.533 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-27T05:13:54.539 INFO:teuthology.task.internal:Opening connections... 2024-04-27T05:13:54.540 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-04-27T05:13:54.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T05:13:54.620 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:13:54.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T05:13:54.695 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-27T05:13:54.702 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-04-27T05:13:54.720 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-04-27T05:13:54.721 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-04-27T05:13:54.778 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-04-27T05:13:54.779 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-27T05:13:54.779 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-04-27T05:13:54.779 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T05:13:54.779 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-27T05:13:54.779 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T05:13:54.779 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-04-27T05:13:54.806 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-27T05:13:54.824 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-27T05:13:54.824 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-27T05:13:54.883 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-27T05:13:54.884 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T05:13:54.885 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-27T05:13:54.905 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-27T05:13:54.914 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-27T05:13:54.919 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-27T05:13:54.919 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-04-27T05:13:54.922 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-27T05:13:54.939 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-27T05:13:54.946 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-27T05:13:54.946 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-04-27T05:13:54.979 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-27T05:13:55.001 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-27T05:13:55.046 INFO:teuthology.run_tasks:Running task kernel... 2024-04-27T05:13:55.061 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-27T05:13:55.061 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-27T05:13:55.062 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T05:13:55.062 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T05:13:55.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:13:55.078 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-04-27T05:13:55.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:13:55.079 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-27T05:13:55.134 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-437.el9.x86_64 2024-04-27T05:13:55.135 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-437.el9.x86_64 2024-04-27T05:13:55.135 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-04-27T05:13:55.136 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-04-27T05:13:55.136 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-437.el9.x86_64 2024-04-27T05:13:55.136 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-04-27T05:13:57.321 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-27T05:13:57.614 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-04-27T05:13:59.097 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-27T05:14:03.506 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.2 MB 00:00 2024-04-27T05:14:05.132 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-27T05:14:10.252 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 21 MB 00:05 2024-04-27T05:14:15.349 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-27T05:14:17.247 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-27T05:14:17.247 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-27T05:14:17.339 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:14:17.340 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:14:17.340 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:14:17.444 DEBUG:teuthology.orchestra.run.smithi114:> echo no | sudo yum reinstall kernel || true 2024-04-27T05:14:17.940 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:02 ago on Sat 27 Apr 2024 05:14:15 AM UTC. 2024-04-27T05:14:18.059 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:14:18.074 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:14:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-04-27T05:14:18.074 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:14:18.074 INFO:teuthology.orchestra.run.smithi114.stdout:Reinstalling: 2024-04-27T05:14:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T05:14:18.074 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:14:18.075 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:14:18.075 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:14:18.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:14:18.075 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 255 k 2024-04-27T05:14:18.075 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 0 2024-04-27T05:14:18.075 INFO:teuthology.orchestra.run.smithi114.stderr:Operation aborted. 2024-04-27T05:14:18.128 INFO:teuthology.orchestra.run.smithi114.stdout:Is this ok [y/N]: 2024-04-27T05:14:18.129 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum reinstall -y kernel || true 2024-04-27T05:14:18.625 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 05:14:15 AM UTC. 2024-04-27T05:14:18.740 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:14:18.740 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:14:18.740 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout:Reinstalling: 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:14:18.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:14:18.742 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 255 k 2024-04-27T05:14:18.742 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 0 2024-04-27T05:14:18.742 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:14:19.235 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.2 MB/s | 255 kB 00:00 2024-04-27T05:14:19.235 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:14:19.235 INFO:teuthology.orchestra.run.smithi114.stdout:Total 516 kB/s | 255 kB 00:00 2024-04-27T05:14:19.257 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:14:19.450 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:14:19.450 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:14:19.455 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:14:19.456 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:14:19.577 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:14:19.669 INFO:teuthology.orchestra.run.smithi114.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T05:14:19.669 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T05:14:19.745 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 925 kB/s | 19 MB 00:21 2024-04-27T05:14:20.385 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T05:14:20.385 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T05:14:21.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T05:14:21.057 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:14:21.057 INFO:teuthology.orchestra.run.smithi114.stdout:Reinstalled: 2024-04-27T05:14:21.057 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-27T05:14:21.058 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:14:21.058 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:14:21.221 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T05:14:21.268 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-27T05:14:21.268 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-27T05:14:21.268 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-27T05:14:21.268 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-27T05:14:21.269 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T05:14:21.306 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-04-27T05:14:21.335 DEBUG:teuthology.parallel:result is None 2024-04-27T05:14:24.223 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-27T05:14:25.929 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-27T05:14:27.942 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-27T05:14:32.948 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-27T05:14:33.129 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:01 ago on Sat 27 Apr 2024 05:14:32 AM UTC. 2024-04-27T05:14:34.821 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-27T05:14:34.821 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-27T05:14:34.915 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:14:34.916 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:14:34.916 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:14:35.021 DEBUG:teuthology.orchestra.run.smithi138:> echo no | sudo yum reinstall kernel || true 2024-04-27T05:14:35.518 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 05:14:32 AM UTC. 2024-04-27T05:14:35.637 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:14:35.653 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalling: 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 255 k 2024-04-27T05:14:35.654 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 0 2024-04-27T05:14:35.655 INFO:teuthology.orchestra.run.smithi138.stderr:Operation aborted. 2024-04-27T05:14:35.711 INFO:teuthology.orchestra.run.smithi138.stdout:Is this ok [y/N]: 2024-04-27T05:14:35.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum reinstall -y kernel || true 2024-04-27T05:14:36.207 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:04 ago on Sat 27 Apr 2024 05:14:32 AM UTC. 2024-04-27T05:14:36.323 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:14:36.324 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:14:36.324 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T05:14:36.324 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalling: 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 255 k 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 0 2024-04-27T05:14:36.325 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:14:36.741 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-437.el9.x86_64.rpm 978 kB/s | 255 kB 00:00 2024-04-27T05:14:36.742 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:14:36.742 INFO:teuthology.orchestra.run.smithi138.stdout:Total 611 kB/s | 255 kB 00:00 2024-04-27T05:14:36.764 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:14:36.961 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:14:36.961 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:14:36.966 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:14:36.966 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:14:37.089 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:14:37.159 INFO:teuthology.orchestra.run.smithi138.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T05:14:37.159 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T05:14:37.894 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T05:14:37.894 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T05:14:38.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T05:14:38.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:14:38.545 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalled: 2024-04-27T05:14:38.545 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-27T05:14:38.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:14:38.545 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:14:38.704 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T05:14:38.751 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-27T05:14:38.751 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-27T05:14:38.751 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-27T05:14:38.751 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-27T05:14:38.751 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T05:14:38.778 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-27T05:14:38.805 DEBUG:teuthology.parallel:result is None 2024-04-27T05:14:38.806 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-27T05:14:38.813 INFO:teuthology.task.internal:Creating test directory... 2024-04-27T05:14:38.813 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T05:14:38.816 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T05:14:38.860 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-27T05:14:38.867 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-27T05:14:38.875 INFO:teuthology.task.internal:Creating archive directory... 2024-04-27T05:14:38.875 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T05:14:38.877 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T05:14:38.978 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-27T05:14:38.984 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-27T05:14:38.984 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T05:14:38.987 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T05:14:39.018 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T05:14:39.023 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T05:14:39.027 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T05:14:39.034 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T05:14:39.035 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-27T05:14:39.042 INFO:teuthology.task.internal:Configuring sudo... 2024-04-27T05:14:39.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T05:14:39.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T05:14:39.102 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-27T05:14:39.112 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-27T05:14:39.112 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T05:14:39.145 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T05:14:39.160 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T05:14:39.243 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T05:14:39.288 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T05:14:39.371 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T05:14:39.411 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:14:39.411 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T05:14:39.474 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T05:14:39.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T05:14:39.569 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T05:14:39.650 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T05:14:39.689 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:14:39.689 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T05:14:39.752 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-04-27T05:14:39.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-27T05:14:39.789 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T05:14:39.837 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T05:14:40.183 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-27T05:14:40.192 INFO:teuthology.task.internal:Starting timer... 2024-04-27T05:14:40.192 INFO:teuthology.run_tasks:Running task pcp... 2024-04-27T05:14:40.202 INFO:teuthology.run_tasks:Running task selinux... 2024-04-27T05:14:40.210 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-04-27T05:14:40.271 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-04-27T05:14:40.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-27T05:14:40.330 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-27T05:14:40.331 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-27T05:14:40.332 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-04-27T05:14:40.352 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-04-27T05:14:40.352 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-27T05:14:40.376 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-27T05:14:40.376 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-04-27T05:14:40.376 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T05:14:40.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:14:40.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T05:14:40.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:14:40.441 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-27T05:14:40.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-04-27T05:14:40.480 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-27T05:14:40.512 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-27T05:14:40.522 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-27T05:14:40.539 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-27T05:14:40.540 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-27T05:18:19.263 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-27T05:18:19.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-27T05:18:19.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T05:18:19.345 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-04-27T05:18:19.421 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-27T05:18:19.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T05:18:19.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T05:18:19.496 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-27T05:18:19.584 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-27T05:18:19.584 INFO:teuthology.run_tasks:Running task clock... 2024-04-27T05:18:19.594 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-27T05:18:19.595 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T05:18:19.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T05:18:19.597 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T05:18:19.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T05:18:19.622 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T05:18:19.635 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T05:18:19.652 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T05:18:19.657 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-04-27T05:18:19.665 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T05:18:19.667 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-04-27T05:18:19.680 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T05:18:19.687 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-27T05:18:19.693 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T05:18:19.698 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-27T05:18:19.710 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T05:18:19.723 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T05:18:19.749 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-04-27T05:18:19.752 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T05:18:19.752 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-04-27T05:18:19.752 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.752 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.753 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.753 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-27T05:18:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T05:18:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-27T05:18:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.779 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:18:19.780 INFO:teuthology.run_tasks:Running task install... 2024-04-27T05:18:19.788 DEBUG:teuthology.task.install:project ceph 2024-04-27T05:18:19.789 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6'}} 2024-04-27T05:18:19.789 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6'} 2024-04-27T05:18:19.789 INFO:teuthology.task.install:Using flavor: default 2024-04-27T05:18:19.795 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-27T05:18:19.795 INFO:teuthology.task.install:extra packages: [] 2024-04-27T05:18:19.795 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'tag': None, 'wait_for_package': False} 2024-04-27T05:18:19.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T05:18:19.798 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'tag': None, 'wait_for_package': False} 2024-04-27T05:18:19.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T05:18:19.943 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/ 2024-04-27T05:18:19.944 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1258.g7d2137f0 2024-04-27T05:18:20.015 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/ 2024-04-27T05:18:20.015 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1258.g7d2137f0 2024-04-27T05:18:20.055 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T05:18:20.055 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:18:20.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T05:18:20.080 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T05:18:20.080 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d2137f0a666c92a9411c1af5c33e4968be54da6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T05:18:20.122 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T05:18:20.122 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:18:20.122 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T05:18:20.147 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T05:18:20.147 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d2137f0a666c92a9411c1af5c33e4968be54da6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T05:18:20.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T05:18:20.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T05:18:20.220 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T05:18:20.283 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-27T05:18:20.284 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T05:18:20.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-27T05:18:20.347 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-04-27T05:18:20.348 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-04-27T05:18:20.508 INFO:teuthology.orchestra.run.smithi138.stdout:48 files removed 2024-04-27T05:18:20.535 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-04-27T05:18:20.572 INFO:teuthology.orchestra.run.smithi114.stdout:48 files removed 2024-04-27T05:18:20.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw 2024-04-27T05:18:21.143 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 217 kB/s | 77 kB 00:00 2024-04-27T05:18:21.165 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 249 kB/s | 77 kB 00:00 2024-04-27T05:18:21.544 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-04-27T05:18:21.591 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-27T05:18:21.970 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-27T05:18:22.100 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-27T05:18:22.750 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-27T05:18:23.362 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-04-27T05:18:24.568 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-27T05:18:25.313 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-27T05:18:29.035 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-04-27T05:18:29.822 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-27T05:18:30.486 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-27T05:18:31.348 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-27T05:18:32.281 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-27T05:18:32.710 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-27T05:18:37.404 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-27T05:18:37.900 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 9.0 kB/s | 1.7 kB 00:00 2024-04-27T05:18:39.537 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:18:39.539 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 7.7 M 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.3 M 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.0 M 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 5.1 M 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 18 M 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 25 k 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-27T05:18:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 708 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 474 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 4.5 M 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 45 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 129 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 161 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-27T05:18:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 321 k 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 298 k 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 99 k 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout:Install 28 Packages 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 2 Packages 2024-04-27T05:18:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 53 M 2024-04-27T05:18:39.543 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:18:40.049 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 7.7 M 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-27T05:18:40.051 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.3 M 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.0 M 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 5.1 M 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 18 M 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 25 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 708 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-27T05:18:40.052 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 474 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 4.5 M 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 45 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 129 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 161 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 321 k 2024-04-27T05:18:40.053 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 298 k 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 99 k 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout:Install 28 Packages 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-04-27T05:18:40.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:18:40.055 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 53 M 2024-04-27T05:18:40.055 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:18:41.663 INFO:teuthology.orchestra.run.smithi114.stdout:(1/30): ceph-common-18.2.2-1258.g7d2137f0.el8.x 12 MB/s | 18 MB 00:01 2024-04-27T05:18:41.763 INFO:teuthology.orchestra.run.smithi114.stdout:(2/30): ceph-base-18.2.2-1258.g7d2137f0.el8.x86 3.1 MB/s | 5.1 MB 00:01 2024-04-27T05:18:41.780 INFO:teuthology.orchestra.run.smithi114.stdout:(3/30): ceph-selinux-18.2.2-1258.g7d2137f0.el8. 215 kB/s | 25 kB 00:00 2024-04-27T05:18:41.872 INFO:teuthology.orchestra.run.smithi114.stdout:(4/30): ceph-radosgw-18.2.2-1258.g7d2137f0.el8. 4.4 MB/s | 7.7 MB 00:01 2024-04-27T05:18:41.906 INFO:teuthology.orchestra.run.smithi114.stdout:(5/30): libradosstriper1-18.2.2-1258.g7d2137f0. 3.7 MB/s | 474 kB 00:00 2024-04-27T05:18:41.956 INFO:teuthology.orchestra.run.smithi114.stdout:(6/30): libcephfs2-18.2.2-1258.g7d2137f0.el8.x8 3.6 MB/s | 708 kB 00:00 2024-04-27T05:18:41.981 INFO:teuthology.orchestra.run.smithi114.stdout:(7/30): python3-ceph-argparse-18.2.2-1258.g7d21 599 kB/s | 45 kB 00:00 2024-04-27T05:18:42.031 INFO:teuthology.orchestra.run.smithi114.stdout:(8/30): python3-ceph-common-18.2.2-1258.g7d2137 1.7 MB/s | 129 kB 00:00 2024-04-27T05:18:42.057 INFO:teuthology.orchestra.run.smithi114.stdout:(9/30): python3-cephfs-18.2.2-1258.g7d2137f0.el 2.1 MB/s | 161 kB 00:00 2024-04-27T05:18:42.140 INFO:teuthology.orchestra.run.smithi114.stdout:(10/30): python3-rbd-18.2.2-1258.g7d2137f0.el8. 3.5 MB/s | 298 kB 00:00 2024-04-27T05:18:42.174 INFO:teuthology.orchestra.run.smithi114.stdout:(11/30): python3-rados-18.2.2-1258.g7d2137f0.el 2.2 MB/s | 321 kB 00:00 2024-04-27T05:18:42.208 INFO:teuthology.orchestra.run.smithi114.stdout:(12/30): python3-rgw-18.2.2-1258.g7d2137f0.el8. 1.5 MB/s | 99 kB 00:00 2024-04-27T05:18:42.484 INFO:teuthology.orchestra.run.smithi138.stdout:(1/30): ceph-radosgw-18.2.2-1258.g7d2137f0.el8. 3.8 MB/s | 7.7 MB 00:02 2024-04-27T05:18:42.526 INFO:teuthology.orchestra.run.smithi138.stdout:(2/30): ceph-selinux-18.2.2-1258.g7d2137f0.el8. 606 kB/s | 25 kB 00:00 2024-04-27T05:18:42.624 INFO:teuthology.orchestra.run.smithi114.stdout:(13/30): librgw2-18.2.2-1258.g7d2137f0.el8.x86_ 5.9 MB/s | 4.5 MB 00:00 2024-04-27T05:18:42.659 INFO:teuthology.orchestra.run.smithi138.stdout:(3/30): libcephfs2-18.2.2-1258.g7d2137f0.el8.x8 5.2 MB/s | 708 kB 00:00 2024-04-27T05:18:42.801 INFO:teuthology.orchestra.run.smithi138.stdout:(4/30): ceph-base-18.2.2-1258.g7d2137f0.el8.x86 2.2 MB/s | 5.1 MB 00:02 2024-04-27T05:18:42.826 INFO:teuthology.orchestra.run.smithi138.stdout:(5/30): libradosstriper1-18.2.2-1258.g7d2137f0. 2.8 MB/s | 474 kB 00:00 2024-04-27T05:18:42.885 INFO:teuthology.orchestra.run.smithi138.stdout:(6/30): python3-ceph-argparse-18.2.2-1258.g7d21 772 kB/s | 45 kB 00:00 2024-04-27T05:18:42.952 INFO:teuthology.orchestra.run.smithi138.stdout:(7/30): python3-ceph-common-18.2.2-1258.g7d2137 1.9 MB/s | 129 kB 00:00 2024-04-27T05:18:43.019 INFO:teuthology.orchestra.run.smithi138.stdout:(8/30): python3-cephfs-18.2.2-1258.g7d2137f0.el 2.4 MB/s | 161 kB 00:00 2024-04-27T05:18:43.058 INFO:teuthology.orchestra.run.smithi114.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 121 kB/s | 107 kB 00:00 2024-04-27T05:18:43.116 INFO:teuthology.orchestra.run.smithi114.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 176 kB/s | 160 kB 00:00 2024-04-27T05:18:43.203 INFO:teuthology.orchestra.run.smithi138.stdout:(9/30): ceph-common-18.2.2-1258.g7d2137f0.el8.x 6.7 MB/s | 18 MB 00:02 2024-04-27T05:18:43.242 INFO:teuthology.orchestra.run.smithi114.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-04-27T05:18:43.251 INFO:teuthology.orchestra.run.smithi138.stdout:(10/30): python3-rados-18.2.2-1258.g7d2137f0.el 1.3 MB/s | 321 kB 00:00 2024-04-27T05:18:43.278 INFO:teuthology.orchestra.run.smithi138.stdout:(11/30): python3-rbd-18.2.2-1258.g7d2137f0.el8. 3.9 MB/s | 298 kB 00:00 2024-04-27T05:18:43.320 INFO:teuthology.orchestra.run.smithi138.stdout:(12/30): python3-rgw-18.2.2-1258.g7d2137f0.el8. 1.4 MB/s | 99 kB 00:00 2024-04-27T05:18:43.571 INFO:teuthology.orchestra.run.smithi138.stdout:(13/30): librgw2-18.2.2-1258.g7d2137f0.el8.x86_ 5.8 MB/s | 4.5 MB 00:00 2024-04-27T05:18:43.621 INFO:teuthology.orchestra.run.smithi138.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 312 kB/s | 107 kB 00:00 2024-04-27T05:18:43.680 INFO:teuthology.orchestra.run.smithi138.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 415 kB/s | 45 kB 00:00 2024-04-27T05:18:43.707 INFO:teuthology.orchestra.run.smithi138.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 416 kB/s | 160 kB 00:00 2024-04-27T05:18:43.830 INFO:teuthology.orchestra.run.smithi138.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-27T05:18:43.881 INFO:teuthology.orchestra.run.smithi138.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-04-27T05:18:43.906 INFO:teuthology.orchestra.run.smithi138.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 211 kB/s | 42 kB 00:00 2024-04-27T05:18:43.925 INFO:teuthology.orchestra.run.smithi114.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 62 kB/s | 42 kB 00:00 2024-04-27T05:18:44.126 INFO:teuthology.orchestra.run.smithi114.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 554 kB/s | 111 kB 00:00 2024-04-27T05:18:44.156 INFO:teuthology.orchestra.run.smithi138.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 341 kB/s | 111 kB 00:00 2024-04-27T05:18:44.176 INFO:teuthology.orchestra.run.smithi114.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 276 kB/s | 292 kB 00:01 2024-04-27T05:18:44.215 INFO:teuthology.orchestra.run.smithi138.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 926 kB/s | 309 kB 00:00 2024-04-27T05:18:44.249 INFO:teuthology.orchestra.run.smithi138.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-04-27T05:18:44.251 INFO:teuthology.orchestra.run.smithi114.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.4 MB/s | 309 kB 00:00 2024-04-27T05:18:44.274 INFO:teuthology.orchestra.run.smithi138.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-04-27T05:18:44.299 INFO:teuthology.orchestra.run.smithi138.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-27T05:18:44.310 INFO:teuthology.orchestra.run.smithi114.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-27T05:18:44.369 INFO:teuthology.orchestra.run.smithi114.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 827 kB/s | 49 kB 00:00 2024-04-27T05:18:44.399 INFO:teuthology.orchestra.run.smithi138.stdout:(25/30): libarrow-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 4.4 MB 00:00 2024-04-27T05:18:44.427 INFO:teuthology.orchestra.run.smithi114.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-27T05:18:44.449 INFO:teuthology.orchestra.run.smithi138.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-04-27T05:18:44.475 INFO:teuthology.orchestra.run.smithi138.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-27T05:18:44.525 INFO:teuthology.orchestra.run.smithi138.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-27T05:18:44.561 INFO:teuthology.orchestra.run.smithi114.stdout:(24/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-04-27T05:18:44.620 INFO:teuthology.orchestra.run.smithi114.stdout:(25/30): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-27T05:18:44.645 INFO:teuthology.orchestra.run.smithi114.stdout:(26/30): librdkafka-1.6.1-102.el9.x86_64.rpm 417 kB/s | 662 kB 00:01 2024-04-27T05:18:44.670 INFO:teuthology.orchestra.run.smithi114.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-27T05:18:44.712 INFO:teuthology.orchestra.run.smithi114.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-27T05:18:44.750 INFO:teuthology.orchestra.run.smithi138.stdout:(29/30): librbd1-18.2.2-1258.g7d2137f0.el8.x86_ 11 MB/s | 3.0 MB 00:00 2024-04-27T05:18:44.971 INFO:teuthology.orchestra.run.smithi114.stdout:(29/30): librbd1-18.2.2-1258.g7d2137f0.el8.x86_ 10 MB/s | 3.0 MB 00:00 2024-04-27T05:18:45.226 INFO:teuthology.orchestra.run.smithi138.stdout:(30/30): librados2-18.2.2-1258.g7d2137f0.el8.x8 4.4 MB/s | 3.3 MB 00:00 2024-04-27T05:18:45.234 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:18:45.234 INFO:teuthology.orchestra.run.smithi138.stdout:Total 10 MB/s | 53 MB 00:05 2024-04-27T05:18:45.308 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:18:45.344 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:18:45.344 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:18:45.704 INFO:teuthology.orchestra.run.smithi114.stdout:(30/30): librados2-18.2.2-1258.g7d2137f0.el8.x8 3.1 MB/s | 3.3 MB 00:01 2024-04-27T05:18:45.706 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:18:45.706 INFO:teuthology.orchestra.run.smithi114.stdout:Total 8.6 MB/s | 53 MB 00:06 2024-04-27T05:18:45.764 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:18:45.765 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:18:45.772 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:18:45.807 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:18:46.226 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:18:46.227 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:18:47.132 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:18:47.218 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-27T05:18:47.247 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-27T05:18:47.296 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-27T05:18:47.585 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:18:47.607 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-27T05:18:47.663 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-27T05:18:47.692 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-27T05:18:47.713 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T05:18:47.741 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-27T05:18:47.805 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T05:18:47.889 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/32 2024-04-27T05:18:47.923 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/32 2024-04-27T05:18:47.994 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/32 2024-04-27T05:18:48.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-27T05:18:48.053 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-27T05:18:48.074 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 9/32 2024-04-27T05:18:48.140 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 10/32 2024-04-27T05:18:48.167 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 11/32 2024-04-27T05:18:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T05:18:48.217 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 11/32 2024-04-27T05:18:48.266 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T05:18:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-27T05:18:48.350 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 6/32 2024-04-27T05:18:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-04-27T05:18:48.376 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/32 2024-04-27T05:18:48.397 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-27T05:18:48.422 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/32 2024-04-27T05:18:48.423 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-27T05:18:48.450 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-04-27T05:18:48.451 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-27T05:18:48.486 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 9/32 2024-04-27T05:18:48.486 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-27T05:18:48.519 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-27T05:18:48.552 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 10/32 2024-04-27T05:18:48.578 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 11/32 2024-04-27T05:18:48.644 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 11/32 2024-04-27T05:18:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-27T05:18:48.785 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-04-27T05:18:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-27T05:18:48.825 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-27T05:18:48.850 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-27T05:18:48.865 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/32 2024-04-27T05:18:48.877 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-04-27T05:18:48.913 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-27T05:18:48.946 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-27T05:18:48.955 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/32 2024-04-27T05:18:48.995 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 21/32 2024-04-27T05:18:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-27T05:18:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-27T05:18:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/32 2024-04-27T05:18:49.383 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 20/32 2024-04-27T05:18:49.413 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 21/32 2024-04-27T05:18:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-04-27T05:18:49.449 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-27T05:18:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 24/32 2024-04-27T05:18:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 24/32 2024-04-27T05:18:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 25/32 2024-04-27T05:18:49.629 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 26/32 2024-04-27T05:18:49.850 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-04-27T05:18:49.878 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 24/32 2024-04-27T05:18:49.941 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 24/32 2024-04-27T05:18:50.017 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 25/32 2024-04-27T05:18:50.047 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 26/32 2024-04-27T05:18:51.802 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 27/32 2024-04-27T05:18:51.851 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 27/32 2024-04-27T05:18:52.149 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 27/32 2024-04-27T05:18:52.368 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 27/32 2024-04-27T05:18:52.404 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T05:18:52.419 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 27/32 2024-04-27T05:18:52.517 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T05:18:52.517 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-27T05:18:52.517 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-27T05:18:52.517 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:18:52.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T05:18:52.951 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 27/32 2024-04-27T05:18:52.989 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T05:18:53.090 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 28/32 2024-04-27T05:18:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-27T05:18:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-27T05:18:53.090 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:18:53.124 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T05:19:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T05:19:02.016 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-27T05:19:02.016 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-27T05:19:02.016 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-27T05:19:02.016 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-27T05:19:02.016 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-27T05:19:02.017 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-27T05:19:02.017 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-27T05:19:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:02.172 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-04-27T05:19:02.173 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:02.739 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T05:19:02.763 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T05:19:02.763 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:02.763 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-27T05:19:02.763 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T05:19:02.764 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T05:19:02.764 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:02.765 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-27T05:19:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-27T05:19:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-27T05:19:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T05:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/32 2024-04-27T05:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-27T05:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T05:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T05:19:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-27T05:19:03.009 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-27T05:19:03.009 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-27T05:19:04.081 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-27T05:19:04.081 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/32 2024-04-27T05:19:04.081 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/32 2024-04-27T05:19:04.081 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/32 2024-04-27T05:19:04.081 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/32 2024-04-27T05:19:04.081 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 6/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 8/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 9/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 10/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 11/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 12/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 13/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-27T05:19:04.082 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-27T05:19:04.083 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T05:19:04.084 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-27T05:19:04.084 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/32 2024-04-27T05:19:04.091 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_6 6/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 7/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8. 8/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x8 9/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 10/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 11/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 12/32 2024-04-27T05:19:04.092 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 13/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-27T05:19:04.093 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-27T05:19:04.094 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/32 2024-04-27T05:19:04.094 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-27T05:19:04.094 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/32 2024-04-27T05:19:04.859 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-27T05:19:04.859 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:04.859 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-27T05:19:04.860 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-27T05:19:04.861 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-04-27T05:19:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-27T05:19:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:04.862 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-27T05:19:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-3.el9.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:04.877 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:05.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-04-27T05:19:05.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-test 2024-04-27T05:19:05.679 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:28 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:05.760 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:28 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:05.801 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 40 M 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 166 k 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-27T05:19:05.802 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:05.803 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:05.803 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:05.803 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-04-27T05:19:05.803 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:05.803 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 40 M 2024-04-27T05:19:05.804 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 163 M 2024-04-27T05:19:05.804 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:05.887 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:05.887 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:05.887 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:05.887 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:05.887 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 40 M 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 166 k 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-04-27T05:19:05.888 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:05.889 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 40 M 2024-04-27T05:19:05.889 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 163 M 2024-04-27T05:19:05.889 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:06.056 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-27T05:19:06.094 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): libcephsqlite-18.2.2-1258.g7d2137f0.el8. 756 kB/s | 166 kB 00:00 2024-04-27T05:19:06.136 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 802 kB/s | 64 kB 00:00 2024-04-27T05:19:06.183 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): libcephsqlite-18.2.2-1258.g7d2137f0.el8. 737 kB/s | 166 kB 00:00 2024-04-27T05:19:06.865 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-04-27T05:19:06.991 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 295 kB/s | 305 kB 00:01 2024-04-27T05:19:08.720 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): ceph-test-18.2.2-1258.g7d2137f0.el8.x86_ 14 MB/s | 40 MB 00:02 2024-04-27T05:19:08.722 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:08.722 INFO:teuthology.orchestra.run.smithi114.stdout:Total 14 MB/s | 40 MB 00:02 2024-04-27T05:19:08.748 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:08.768 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:08.768 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:09.081 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:09.082 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:09.482 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): ceph-test-18.2.2-1258.g7d2137f0.el8.x86_ 11 MB/s | 40 MB 00:03 2024-04-27T05:19:09.484 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:09.484 INFO:teuthology.orchestra.run.smithi138.stdout:Total 11 MB/s | 40 MB 00:03 2024-04-27T05:19:09.511 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:09.531 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:09.531 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:09.547 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:09.622 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-27T05:19:09.669 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-27T05:19:09.697 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/4 2024-04-27T05:19:09.844 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:09.845 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:10.313 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:10.407 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-27T05:19:10.456 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-27T05:19:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/4 2024-04-27T05:19:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/4 2024-04-27T05:19:13.452 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/4 2024-04-27T05:19:14.143 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/4 2024-04-27T05:19:14.221 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/4 2024-04-27T05:19:15.476 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/4 2024-04-27T05:19:15.476 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/4 2024-04-27T05:19:15.477 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/4 2024-04-27T05:19:15.477 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:15.738 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:16.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph 2024-04-27T05:19:16.307 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/4 2024-04-27T05:19:16.307 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/4 2024-04-27T05:19:16.307 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/4 2024-04-27T05:19:16.307 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-27T05:19:16.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:16.591 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:16.594 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:39 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:16.717 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:16.719 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:16.719 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:16.719 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:16.719 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:16.719 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 6.5 k 2024-04-27T05:19:16.719 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 2.1 M 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 1.5 M 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 246 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 4.7 M 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 17 M 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-27T05:19:16.720 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-27T05:19:16.721 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout:Install 33 Packages 2024-04-27T05:19:16.722 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:16.723 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 29 M 2024-04-27T05:19:16.723 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 104 M 2024-04-27T05:19:16.723 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:16.874 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-04-27T05:19:17.384 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:40 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:17.510 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:17.512 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:17.512 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:17.512 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:17.512 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:17.512 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 6.5 k 2024-04-27T05:19:17.512 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 2.1 M 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 1.5 M 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 246 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 4.7 M 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 17 M 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-27T05:19:17.513 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-27T05:19:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout:Install 33 Packages 2024-04-27T05:19:17.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:17.516 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 29 M 2024-04-27T05:19:17.516 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 104 M 2024-04-27T05:19:17.516 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:17.554 INFO:teuthology.orchestra.run.smithi114.stdout:(1/33): ceph-18.2.2-1258.g7d2137f0.el8.x86_64.r 40 kB/s | 6.5 kB 00:00 2024-04-27T05:19:17.787 INFO:teuthology.orchestra.run.smithi114.stdout:(2/33): ceph-mds-18.2.2-1258.g7d2137f0.el8.x86_ 5.3 MB/s | 2.1 MB 00:00 2024-04-27T05:19:18.238 INFO:teuthology.orchestra.run.smithi114.stdout:(3/33): ceph-mgr-18.2.2-1258.g7d2137f0.el8.x86_ 1.7 MB/s | 1.5 MB 00:00 2024-04-27T05:19:18.304 INFO:teuthology.orchestra.run.smithi114.stdout:(4/33): ceph-mon-18.2.2-1258.g7d2137f0.el8.x86_ 6.2 MB/s | 4.7 MB 00:00 2024-04-27T05:19:18.355 INFO:teuthology.orchestra.run.smithi114.stdout:(5/33): ceph-mgr-modules-core-18.2.2-1258.g7d21 2.1 MB/s | 246 kB 00:00 2024-04-27T05:19:18.447 INFO:teuthology.orchestra.run.smithi114.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-27T05:19:18.488 INFO:teuthology.orchestra.run.smithi114.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-27T05:19:18.530 INFO:teuthology.orchestra.run.smithi114.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-04-27T05:19:18.706 INFO:teuthology.orchestra.run.smithi114.stdout:(9/33): ceph-osd-18.2.2-1258.g7d2137f0.el8.x86_ 18 MB/s | 17 MB 00:00 2024-04-27T05:19:18.739 INFO:teuthology.orchestra.run.smithi114.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 3.2 MB/s | 1.2 MB 00:00 2024-04-27T05:19:18.764 INFO:teuthology.orchestra.run.smithi114.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 533 kB/s | 125 kB 00:00 2024-04-27T05:19:18.873 INFO:teuthology.orchestra.run.smithi114.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-04-27T05:19:18.898 INFO:teuthology.orchestra.run.smithi114.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.1 MB/s | 215 kB 00:00 2024-04-27T05:19:18.933 INFO:teuthology.orchestra.run.smithi138.stdout:(1/33): ceph-18.2.2-1258.g7d2137f0.el8.x86_64.r 37 kB/s | 6.5 kB 00:00 2024-04-27T05:19:18.949 INFO:teuthology.orchestra.run.smithi114.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-04-27T05:19:18.974 INFO:teuthology.orchestra.run.smithi114.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 735 kB/s | 172 kB 00:00 2024-04-27T05:19:18.999 INFO:teuthology.orchestra.run.smithi114.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-27T05:19:19.124 INFO:teuthology.orchestra.run.smithi114.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 247 kB/s | 43 kB 00:00 2024-04-27T05:19:19.158 INFO:teuthology.orchestra.run.smithi114.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 331 kB/s | 11 kB 00:00 2024-04-27T05:19:19.192 INFO:teuthology.orchestra.run.smithi114.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 530 kB/s | 18 kB 00:00 2024-04-27T05:19:19.200 INFO:teuthology.orchestra.run.smithi138.stdout:(2/33): ceph-mds-18.2.2-1258.g7d2137f0.el8.x86_ 4.7 MB/s | 2.1 MB 00:00 2024-04-27T05:19:19.217 INFO:teuthology.orchestra.run.smithi114.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 708 kB/s | 172 kB 00:00 2024-04-27T05:19:19.242 INFO:teuthology.orchestra.run.smithi114.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-04-27T05:19:19.250 INFO:teuthology.orchestra.run.smithi138.stdout:(3/33): ceph-mgr-18.2.2-1258.g7d2137f0.el8.x86_ 3.0 MB/s | 1.5 MB 00:00 2024-04-27T05:19:19.267 INFO:teuthology.orchestra.run.smithi114.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-04-27T05:19:19.293 INFO:teuthology.orchestra.run.smithi114.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-04-27T05:19:19.318 INFO:teuthology.orchestra.run.smithi114.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-04-27T05:19:19.343 INFO:teuthology.orchestra.run.smithi114.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 615 kB/s | 46 kB 00:00 2024-04-27T05:19:19.351 INFO:teuthology.orchestra.run.smithi138.stdout:(4/33): ceph-mgr-modules-core-18.2.2-1258.g7d21 2.4 MB/s | 246 kB 00:00 2024-04-27T05:19:19.368 INFO:teuthology.orchestra.run.smithi114.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-04-27T05:19:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-27T05:19:19.435 INFO:teuthology.orchestra.run.smithi114.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-04-27T05:19:19.461 INFO:teuthology.orchestra.run.smithi114.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 977 kB/s | 90 kB 00:00 2024-04-27T05:19:19.468 INFO:teuthology.orchestra.run.smithi138.stdout:(5/33): ceph-mon-18.2.2-1258.g7d2137f0.el8.x86_ 8.7 MB/s | 4.7 MB 00:00 2024-04-27T05:19:19.486 INFO:teuthology.orchestra.run.smithi114.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 476 kB/s | 36 kB 00:00 2024-04-27T05:19:19.511 INFO:teuthology.orchestra.run.smithi114.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-27T05:19:19.545 INFO:teuthology.orchestra.run.smithi114.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-04-27T05:19:19.568 INFO:teuthology.orchestra.run.smithi138.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-27T05:19:19.570 INFO:teuthology.orchestra.run.smithi114.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 240 kB/s | 20 kB 00:00 2024-04-27T05:19:19.574 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:19.574 INFO:teuthology.orchestra.run.smithi114.stdout:Total 10 MB/s | 29 MB 00:02 2024-04-27T05:19:19.618 INFO:teuthology.orchestra.run.smithi138.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-27T05:19:19.668 INFO:teuthology.orchestra.run.smithi138.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-04-27T05:19:19.684 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:19.685 INFO:teuthology.orchestra.run.smithi138.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-04-27T05:19:19.711 INFO:teuthology.orchestra.run.smithi138.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.9 MB/s | 125 kB 00:00 2024-04-27T05:19:19.721 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:19.721 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:19.736 INFO:teuthology.orchestra.run.smithi138.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-04-27T05:19:20.001 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:20.001 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:20.079 INFO:teuthology.orchestra.run.smithi138.stdout:(12/33): ceph-osd-18.2.2-1258.g7d2137f0.el8.x86 19 MB/s | 17 MB 00:00 2024-04-27T05:19:20.528 INFO:teuthology.orchestra.run.smithi138.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 44 kB/s | 35 kB 00:00 2024-04-27T05:19:20.703 INFO:teuthology.orchestra.run.smithi138.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 173 kB/s | 172 kB 00:00 2024-04-27T05:19:20.837 INFO:teuthology.orchestra.run.smithi138.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 72 kB/s | 51 kB 00:00 2024-04-27T05:19:21.012 INFO:teuthology.orchestra.run.smithi138.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 141 kB/s | 43 kB 00:00 2024-04-27T05:19:21.146 INFO:teuthology.orchestra.run.smithi138.stdout:(17/33): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-04-27T05:19:21.159 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:21.204 INFO:teuthology.orchestra.run.smithi138.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 468 kB/s | 172 kB 00:00 2024-04-27T05:19:21.231 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-27T05:19:21.263 INFO:teuthology.orchestra.run.smithi138.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-04-27T05:19:21.272 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-27T05:19:21.305 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-27T05:19:21.305 INFO:teuthology.orchestra.run.smithi138.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 236 kB/s | 18 kB 00:00 2024-04-27T05:19:21.330 INFO:teuthology.orchestra.run.smithi138.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-04-27T05:19:21.356 INFO:teuthology.orchestra.run.smithi138.stdout:(22/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 51 kB/s | 11 kB 00:00 2024-04-27T05:19:21.373 INFO:teuthology.orchestra.run.smithi138.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 289 kB/s | 19 kB 00:00 2024-04-27T05:19:21.390 INFO:teuthology.orchestra.run.smithi138.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 333 kB/s | 20 kB 00:00 2024-04-27T05:19:21.397 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-27T05:19:21.473 INFO:teuthology.orchestra.run.smithi138.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 788 kB/s | 79 kB 00:00 2024-04-27T05:19:21.474 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-27T05:19:21.499 INFO:teuthology.orchestra.run.smithi138.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-27T05:19:21.515 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-27T05:19:21.523 INFO:teuthology.orchestra.run.smithi138.stdout:(27/33): python3-logutils-0.3.5-21.el9.noarch.r 276 kB/s | 46 kB 00:00 2024-04-27T05:19:21.549 INFO:teuthology.orchestra.run.smithi138.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 217 kB/s | 16 kB 00:00 2024-04-27T05:19:21.550 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-27T05:19:21.574 INFO:teuthology.orchestra.run.smithi138.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-04-27T05:19:21.580 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-27T05:19:21.615 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-27T05:19:21.641 INFO:teuthology.orchestra.run.smithi138.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 305 kB/s | 36 kB 00:00 2024-04-27T05:19:21.648 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-27T05:19:21.666 INFO:teuthology.orchestra.run.smithi138.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-27T05:19:21.689 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-27T05:19:21.692 INFO:teuthology.orchestra.run.smithi138.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-04-27T05:19:21.717 INFO:teuthology.orchestra.run.smithi138.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 265 kB/s | 20 kB 00:00 2024-04-27T05:19:21.723 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:21.723 INFO:teuthology.orchestra.run.smithi138.stdout:Total 6.9 MB/s | 29 MB 00:04 2024-04-27T05:19:21.724 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-27T05:19:21.767 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-27T05:19:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-27T05:19:21.842 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-27T05:19:21.879 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:21.879 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:22.017 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-27T05:19:22.096 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-27T05:19:22.149 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-27T05:19:22.156 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:22.156 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:22.196 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-27T05:19:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-27T05:19:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-27T05:19:22.488 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-27T05:19:22.542 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-27T05:19:22.619 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-27T05:19:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-27T05:19:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-27T05:19:22.873 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-27T05:19:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 28/33 2024-04-27T05:19:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/33 2024-04-27T05:19:23.084 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/33 2024-04-27T05:19:23.084 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:23.084 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-27T05:19:23.084 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T05:19:23.085 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T05:19:23.085 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-27T05:19:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-27T05:19:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-27T05:19:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-27T05:19:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-27T05:19:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-27T05:19:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-27T05:19:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-27T05:19:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-27T05:19:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-27T05:19:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-27T05:19:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-27T05:19:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-27T05:19:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-27T05:19:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-27T05:19:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-27T05:19:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-27T05:19:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-27T05:19:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-27T05:19:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-27T05:19:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-27T05:19:24.546 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/33 2024-04-27T05:19:24.574 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/33 2024-04-27T05:19:24.574 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:24.574 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-27T05:19:24.574 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T05:19:24.574 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T05:19:24.575 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-27T05:19:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-27T05:19:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-27T05:19:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-27T05:19:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-27T05:19:25.022 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/33 2024-04-27T05:19:25.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-27T05:19:25.046 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/33 2024-04-27T05:19:25.046 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:25.046 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-27T05:19:25.046 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T05:19:25.046 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T05:19:25.046 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:25.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 28/33 2024-04-27T05:19:25.216 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/33 2024-04-27T05:19:25.240 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 29/33 2024-04-27T05:19:25.240 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:25.240 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-27T05:19:25.240 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T05:19:25.241 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T05:19:25.241 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:25.248 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/33 2024-04-27T05:19:25.271 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/33 2024-04-27T05:19:25.271 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:25.271 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-27T05:19:25.271 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T05:19:25.271 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T05:19:25.271 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:25.326 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 33/33 2024-04-27T05:19:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/33 2024-04-27T05:19:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 30/33 2024-04-27T05:19:26.700 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:26.700 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-27T05:19:26.700 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T05:19:26.700 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T05:19:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:27.008 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 33/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 6/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-27T05:19:27.009 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-27T05:19:27.010 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-27T05:19:27.011 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-27T05:19:27.011 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-27T05:19:27.011 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-27T05:19:27.011 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-27T05:19:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/33 2024-04-27T05:19:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 31/33 2024-04-27T05:19:27.193 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:27.193 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-27T05:19:27.193 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T05:19:27.193 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T05:19:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:27.435 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/33 2024-04-27T05:19:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 32/33 2024-04-27T05:19:27.459 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:27.459 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-27T05:19:27.459 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T05:19:27.459 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T05:19:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 33/33 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:27.858 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-27T05:19:27.859 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-27T05:19:27.860 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:27.861 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:28.220 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-base 2024-04-27T05:19:28.734 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:51 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:28.813 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:19:28.853 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:28.854 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:19:28.854 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:28.919 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install cephadm 2024-04-27T05:19:29.256 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 33/33 2024-04-27T05:19:29.256 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 3/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 4/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 5/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8. 6/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-27T05:19:29.257 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-27T05:19:29.258 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-27T05:19:29.259 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-27T05:19:29.259 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-27T05:19:29.259 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-27T05:19:29.259 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-27T05:19:29.260 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-27T05:19:29.260 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-27T05:19:29.260 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-27T05:19:29.424 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:52 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:29.543 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 224 k 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:29.544 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:29.545 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:19:29.545 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:29.545 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 224 k 2024-04-27T05:19:29.545 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 216 k 2024-04-27T05:19:29.545 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:29.921 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm-18.2.2-1258.g7d2137f0.el8.noarch.rpm 595 kB/s | 224 kB 00:00 2024-04-27T05:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout:Total 593 kB/s | 224 kB 00:00 2024-04-27T05:19:29.922 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:29.927 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:29.927 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:29.930 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:29.930 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:29.997 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-27T05:19:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-27T05:19:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:30.079 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:30.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-04-27T05:19:30.432 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:30.514 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:30.841 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:30.934 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:53 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:31.012 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:19:31.053 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:31.054 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:19:31.054 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:31.058 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:31.059 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:31.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-04-27T05:19:31.281 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T05:19:31.616 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:54 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:31.735 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 224 k 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:19:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:31.737 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 224 k 2024-04-27T05:19:31.737 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 216 k 2024-04-27T05:19:31.737 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:31.794 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:54 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:31.915 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:31.916 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-04-27T05:19:31.916 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:31.916 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 142 k 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 142 k 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 395 k 2024-04-27T05:19:31.917 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:31.968 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-18.2.2-1258.g7d2137f0.el8.noarch.rpm 969 kB/s | 224 kB 00:00 2024-04-27T05:19:31.968 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:31.969 INFO:teuthology.orchestra.run.smithi138.stdout:Total 963 kB/s | 224 kB 00:00 2024-04-27T05:19:31.969 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:31.974 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:31.977 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:31.977 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:32.041 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:32.118 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-immutable-object-cache-18.2.2-1258.g7d2137 708 kB/s | 142 kB 00:00 2024-04-27T05:19:32.119 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:32.119 INFO:teuthology.orchestra.run.smithi114.stdout:Total 702 kB/s | 142 kB 00:00 2024-04-27T05:19:32.120 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:32.127 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:32.127 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:32.183 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:32.184 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:32.335 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:32.411 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T05:19:32.435 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T05:19:32.435 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:32.435 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-27T05:19:32.435 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:32.488 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:32.571 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:32.884 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:33.130 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/1 2024-04-27T05:19:33.130 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:33.130 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:33.130 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:33.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:33.131 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:33.267 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T05:19:33.267 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:33.268 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:33.268 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:33.268 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:33.268 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:33.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T05:19:33.460 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr 2024-04-27T05:19:33.839 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:56 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 142 k 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:19:33.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 142 k 2024-04-27T05:19:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 395 k 2024-04-27T05:19:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:33.974 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:56 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:34.056 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:19:34.098 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:34.099 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:19:34.099 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:34.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T05:19:34.267 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-18.2.2-1258.g7d2137 471 kB/s | 142 kB 00:00 2024-04-27T05:19:34.268 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:34.268 INFO:teuthology.orchestra.run.smithi138.stdout:Total 468 kB/s | 142 kB 00:00 2024-04-27T05:19:34.268 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:34.276 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:34.276 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:34.333 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:34.333 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:34.584 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T05:19:34.609 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T05:19:34.610 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:19:34.610 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-27T05:19:34.610 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:34.679 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:57 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:34.808 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 3.5 M 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 24 k 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 15 k 2024-04-27T05:19:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-27T05:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-27T05:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout:Install 5 Packages 2024-04-27T05:19:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 3.8 M 2024-04-27T05:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 87 M 2024-04-27T05:19:34.813 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:35.140 INFO:teuthology.orchestra.run.smithi114.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1258.g7d21 62 kB/s | 15 kB 00:00 2024-04-27T05:19:35.165 INFO:teuthology.orchestra.run.smithi114.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1258.g7d2 90 kB/s | 24 kB 00:00 2024-04-27T05:19:35.357 INFO:teuthology.orchestra.run.smithi114.stdout:(3/5): ceph-mgr-dashboard-18.2.2-1258.g7d2137f0 7.6 MB/s | 3.5 MB 00:00 2024-04-27T05:19:35.382 INFO:teuthology.orchestra.run.smithi114.stdout:(4/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 127 kB/s | 31 kB 00:00 2024-04-27T05:19:35.467 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0. 1/1 2024-04-27T05:19:35.467 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:35.467 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:35.468 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:19:35.468 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:35.468 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:35.491 INFO:teuthology.orchestra.run.smithi114.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 579 kB/s | 188 kB 00:00 2024-04-27T05:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:35.494 INFO:teuthology.orchestra.run.smithi114.stdout:Total 5.5 MB/s | 3.8 MB 00:00 2024-04-27T05:19:35.495 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:35.501 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:35.502 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:35.560 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:35.560 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:35.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-04-27T05:19:35.815 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:35.906 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-04-27T05:19:35.937 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-04-27T05:19:35.966 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T05:19:36.190 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:36.269 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:19:36.312 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:36.313 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:19:36.313 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:36.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T05:19:36.878 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:37.000 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 3.5 M 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:19:37.001 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 24 k 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 15 k 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout:Install 5 Packages 2024-04-27T05:19:37.002 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:37.003 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.8 M 2024-04-27T05:19:37.003 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 87 M 2024-04-27T05:19:37.003 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:37.011 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 4/5 2024-04-27T05:19:37.044 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T05:19:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1258.g7d21 80 kB/s | 15 kB 00:00 2024-04-27T05:19:37.372 INFO:teuthology.orchestra.run.smithi138.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1258.g7d2 83 kB/s | 24 kB 00:00 2024-04-27T05:19:37.530 INFO:teuthology.orchestra.run.smithi138.stdout:(3/5): ceph-mgr-dashboard-18.2.2-1258.g7d2137f0 7.8 MB/s | 3.5 MB 00:00 2024-04-27T05:19:37.556 INFO:teuthology.orchestra.run.smithi138.stdout:(4/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 108 kB/s | 31 kB 00:00 2024-04-27T05:19:37.764 INFO:teuthology.orchestra.run.smithi138.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 481 kB/s | 188 kB 00:00 2024-04-27T05:19:37.766 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:37.767 INFO:teuthology.orchestra.run.smithi138.stdout:Total 4.9 MB/s | 3.8 MB 00:00 2024-04-27T05:19:37.767 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:37.774 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:37.774 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:37.833 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:37.834 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:38.102 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:38.109 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T05:19:38.109 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 1/5 2024-04-27T05:19:38.109 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 2/5 2024-04-27T05:19:38.109 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T05:19:38.109 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-04-27T05:19:38.187 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-04-27T05:19:38.227 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-04-27T05:19:38.256 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:38.331 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:38.542 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T05:19:39.047 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:02 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-04-27T05:19:39.170 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-04-27T05:19:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:39.170 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 7.4 M 2024-04-27T05:19:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:39.171 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-04-27T05:19:39.172 INFO:teuthology.orchestra.run.smithi114.stdout:Install 12 Packages 2024-04-27T05:19:39.172 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:39.172 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 38 M 2024-04-27T05:19:39.172 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 209 M 2024-04-27T05:19:39.172 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:39.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 4/5 2024-04-27T05:19:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T05:19:39.798 INFO:teuthology.orchestra.run.smithi114.stdout:(1/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-27T05:19:39.890 INFO:teuthology.orchestra.run.smithi114.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-04-27T05:19:39.918 INFO:teuthology.orchestra.run.smithi114.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 516 kB/s | 195 kB 00:00 2024-04-27T05:19:39.999 INFO:teuthology.orchestra.run.smithi114.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-04-27T05:19:40.091 INFO:teuthology.orchestra.run.smithi114.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-04-27T05:19:40.107 INFO:teuthology.orchestra.run.smithi114.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-04-27T05:19:40.233 INFO:teuthology.orchestra.run.smithi114.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 33 MB/s | 4.7 MB 00:00 2024-04-27T05:19:40.350 INFO:teuthology.orchestra.run.smithi114.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.2-12 9.1 MB/s | 7.4 MB 00:00 2024-04-27T05:19:40.375 INFO:teuthology.orchestra.run.smithi114.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 913 kB/s | 244 kB 00:00 2024-04-27T05:19:40.475 INFO:teuthology.orchestra.run.smithi114.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-27T05:19:40.488 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 5/5 2024-04-27T05:19:40.488 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8. 1/5 2024-04-27T05:19:40.488 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarc 2/5 2024-04-27T05:19:40.488 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.n 3/5 2024-04-27T05:19:40.488 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-04-27T05:19:40.500 INFO:teuthology.orchestra.run.smithi114.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.6 MB/s | 250 kB 00:00 2024-04-27T05:19:40.729 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-04-27T05:19:40.729 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:40.729 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:40.729 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:40.729 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:40.729 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:40.730 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-27T05:19:40.730 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-27T05:19:40.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:40.730 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:40.909 INFO:teuthology.orchestra.run.smithi114.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-27T05:19:40.910 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:40.910 INFO:teuthology.orchestra.run.smithi114.stdout:Total 22 MB/s | 38 MB 00:01 2024-04-27T05:19:40.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T05:19:41.259 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:41.287 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:41.287 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:41.437 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:04 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:41.559 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:41.560 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-27T05:19:41.560 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:41.560 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-27T05:19:41.560 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:41.560 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 7.4 M 2024-04-27T05:19:41.560 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:41.561 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-27T05:19:41.562 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-04-27T05:19:41.562 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:41.562 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 38 M 2024-04-27T05:19:41.562 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 209 M 2024-04-27T05:19:41.562 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:41.690 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:41.690 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:41.880 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-04-27T05:19:41.939 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-04-27T05:19:42.064 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 178 kB/s | 33 kB 00:00 2024-04-27T05:19:42.123 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-04-27T05:19:42.181 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-04-27T05:19:42.365 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): ceph-mgr-diskprediction-local-18.2.2-12 12 MB/s | 7.4 MB 00:00 2024-04-27T05:19:42.432 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.1 MB/s | 3.0 MB 00:00 2024-04-27T05:19:42.442 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-27T05:19:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-27T05:19:42.624 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 944 kB/s | 244 kB 00:00 2024-04-27T05:19:42.651 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-27T05:19:42.692 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-27T05:19:42.724 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 8.7 MB/s | 4.7 MB 00:00 2024-04-27T05:19:42.749 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-04-27T05:19:42.841 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-27T05:19:42.869 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-27T05:19:42.899 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-27T05:19:42.975 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-27T05:19:43.500 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-27T05:19:43.501 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:43.501 INFO:teuthology.orchestra.run.smithi138.stdout:Total 20 MB/s | 38 MB 00:01 2024-04-27T05:19:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-27T05:19:43.765 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-27T05:19:43.851 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:43.878 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:43.878 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:44.285 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:44.285 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:45.046 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T05:19:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-27T05:19:45.156 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-27T05:19:45.313 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-27T05:19:45.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-27T05:19:45.537 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-27T05:19:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-27T05:19:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-27T05:19:46.064 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-27T05:19:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d213 12/12 2024-04-27T05:19:46.311 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-27T05:19:46.437 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-27T05:19:47.666 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d213 12/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d213 1/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-27T05:19:48.538 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T05:19:48.539 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-27T05:19:48.714 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-27T05:19:48.740 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d213 12/12 2024-04-27T05:19:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-27T05:19:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:48.953 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:48.953 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:48.954 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:49.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-rook 2024-04-27T05:19:49.845 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:12 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:49.965 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:49.966 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:49.966 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:49.966 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:49.966 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 49 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-27T05:19:49.967 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout:Install 13 Packages 2024-04-27T05:19:49.968 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:49.969 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 2.2 M 2024-04-27T05:19:49.969 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 26 M 2024-04-27T05:19:49.969 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:50.489 INFO:teuthology.orchestra.run.smithi114.stdout:(1/13): ceph-mgr-rook-18.2.2-1258.g7d2137f0.el8 160 kB/s | 49 kB 00:00 2024-04-27T05:19:50.706 INFO:teuthology.orchestra.run.smithi114.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 36 kB/s | 19 kB 00:00 2024-04-27T05:19:50.781 INFO:teuthology.orchestra.run.smithi114.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 43 kB/s | 26 kB 00:00 2024-04-27T05:19:51.413 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d213 12/12 2024-04-27T05:19:51.413 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d213 1/12 2024-04-27T05:19:51.413 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-27T05:19:51.413 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-27T05:19:51.413 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T05:19:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-27T05:19:51.423 INFO:teuthology.orchestra.run.smithi114.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 238 kB/s | 222 kB 00:00 2024-04-27T05:19:51.515 INFO:teuthology.orchestra.run.smithi114.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 196 kB/s | 159 kB 00:00 2024-04-27T05:19:51.632 INFO:teuthology.orchestra.run.smithi114.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 277 kB/s | 32 kB 00:00 2024-04-27T05:19:51.707 INFO:teuthology.orchestra.run.smithi114.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 200 kB/s | 15 kB 00:00 2024-04-27T05:19:51.733 INFO:teuthology.orchestra.run.smithi114.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 293 kB/s | 279 kB 00:00 2024-04-27T05:19:51.848 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-27T05:19:51.848 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:51.848 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:51.848 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:51.849 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:51.883 INFO:teuthology.orchestra.run.smithi114.stdout:(9/13): python3-google-auth-2.29.0-1.el9.noarch 1.1 MB/s | 206 kB 00:00 2024-04-27T05:19:51.933 INFO:teuthology.orchestra.run.smithi114.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-27T05:19:52.000 INFO:teuthology.orchestra.run.smithi114.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-27T05:19:52.033 INFO:teuthology.orchestra.run.smithi114.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.4 MB/s | 1.0 MB 00:00 2024-04-27T05:19:52.084 INFO:teuthology.orchestra.run.smithi114.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 81 kB/s | 54 kB 00:00 2024-04-27T05:19:52.084 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:52.085 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.0 MB/s | 2.2 MB 00:02 2024-04-27T05:19:52.146 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:52.160 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:52.161 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:52.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-04-27T05:19:52.244 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:52.246 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:52.746 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:15 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:52.827 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:52.866 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 49 k 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-27T05:19:52.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2024-04-27T05:19:52.869 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:52.870 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.2 M 2024-04-27T05:19:52.870 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-04-27T05:19:52.870 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:52.993 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-27T05:19:53.052 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-27T05:19:53.107 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-27T05:19:53.144 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-27T05:19:53.186 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-27T05:19:53.249 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 143 kB/s | 26 kB 00:00 2024-04-27T05:19:53.266 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 98 kB/s | 19 kB 00:00 2024-04-27T05:19:53.281 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-27T05:19:53.283 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): ceph-mgr-rook-18.2.2-1258.g7d2137f0.el8 231 kB/s | 49 kB 00:00 2024-04-27T05:19:53.367 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-04-27T05:19:53.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-27T05:19:53.392 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-27T05:19:53.434 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 810 kB/s | 54 kB 00:00 2024-04-27T05:19:53.437 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-27T05:19:53.493 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-04-27T05:19:53.560 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2024-04-27T05:19:53.610 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 85 kB/s | 15 kB 00:00 2024-04-27T05:19:53.735 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-04-27T05:19:53.861 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 559 kB/s | 206 kB 00:00 2024-04-27T05:19:53.886 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 596 kB/s | 90 kB 00:00 2024-04-27T05:19:53.919 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.8 MB/s | 1.0 MB 00:00 2024-04-27T05:19:53.922 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:53.922 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-04-27T05:19:53.983 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:19:53.998 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:19:53.998 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:19:54.080 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:19:54.082 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:19:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-27T05:19:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-27T05:19:54.227 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-27T05:19:54.275 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-27T05:19:54.297 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 13/13 2024-04-27T05:19:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:19:54.832 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-27T05:19:54.887 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-27T05:19:54.967 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-27T05:19:54.997 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-27T05:19:55.039 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-27T05:19:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 13/13 2024-04-27T05:19:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 1/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-27T05:19:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-27T05:19:55.131 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-27T05:19:55.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-27T05:19:55.299 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-27T05:19:55.373 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:55.593 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T05:19:56.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-27T05:19:56.056 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-27T05:19:56.090 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-27T05:19:56.102 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:19 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-27T05:19:56.168 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 13/13 2024-04-27T05:19:56.220 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:19:56.221 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 138 k 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:19:56.223 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:19:56.223 INFO:teuthology.orchestra.run.smithi114.stdout:Install 6 Packages 2024-04-27T05:19:56.223 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:56.223 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 7.0 M 2024-04-27T05:19:56.223 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 33 M 2024-04-27T05:19:56.223 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:19:56.921 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 13/13 2024-04-27T05:19:56.921 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 1/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-27T05:19:56.922 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-27T05:19:56.953 INFO:teuthology.orchestra.run.smithi114.stdout:(1/6): ceph-mgr-cephadm-18.2.2-1258.g7d2137f0.e 397 kB/s | 138 kB 00:00 2024-04-27T05:19:56.978 INFO:teuthology.orchestra.run.smithi114.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 664 kB/s | 248 kB 00:00 2024-04-27T05:19:57.053 INFO:teuthology.orchestra.run.smithi114.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 769 kB/s | 58 kB 00:00 2024-04-27T05:19:57.096 INFO:teuthology.orchestra.run.smithi114.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.2 MB/s | 51 kB 00:00 2024-04-27T05:19:57.121 INFO:teuthology.orchestra.run.smithi114.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.2 MB/s | 548 kB 00:00 2024-04-27T05:19:57.237 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-27T05:19:57.238 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-27T05:19:57.239 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-27T05:19:57.239 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-27T05:19:57.239 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-27T05:19:57.239 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:57.239 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:19:57.388 INFO:teuthology.orchestra.run.smithi114.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.6 MB/s | 6.0 MB 00:00 2024-04-27T05:19:57.388 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:19:57.389 INFO:teuthology.orchestra.run.smithi114.stdout:Total 6.0 MB/s | 7.0 MB 00:01 2024-04-27T05:19:57.458 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:19:57.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T05:19:57.468 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:19:57.469 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:19:57.563 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:19:57.563 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:19:57.863 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:19:57.982 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:20 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:19:58.034 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-27T05:19:58.088 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-27T05:19:58.101 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:19:58.102 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:58.102 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:19:58.102 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:58.102 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:19:58.102 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 138 k 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-04-27T05:19:58.103 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:19:58.104 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 7.0 M 2024-04-27T05:19:58.104 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 33 M 2024-04-27T05:19:58.104 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:19:58.596 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-27T05:19:58.682 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-27T05:19:58.734 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): ceph-mgr-cephadm-18.2.2-1258.g7d2137f0.e 392 kB/s | 138 kB 00:00 2024-04-27T05:19:58.745 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-27T05:19:58.768 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 6/6 2024-04-27T05:19:59.317 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 265 kB/s | 248 kB 00:00 2024-04-27T05:19:59.462 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 6/6 2024-04-27T05:19:59.462 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/6 2024-04-27T05:19:59.462 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-27T05:19:59.462 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-27T05:19:59.462 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-27T05:19:59.462 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-27T05:19:59.712 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-27T05:19:59.713 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-27T05:19:59.713 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:19:59.713 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:19:59.867 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 484 kB/s | 548 kB 00:01 2024-04-27T05:19:59.884 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 102 kB/s | 58 kB 00:00 2024-04-27T05:19:59.950 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-fuse 2024-04-27T05:19:59.976 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 470 kB/s | 51 kB 00:00 2024-04-27T05:20:00.465 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:23 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:00.585 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:00.585 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:00.585 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 849 k 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout:Install 2 Packages 2024-04-27T05:20:00.586 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:00.587 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 929 k 2024-04-27T05:20:00.587 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 2.7 M 2024-04-27T05:20:00.587 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:01.014 INFO:teuthology.orchestra.run.smithi114.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 533 kB/s | 80 kB 00:00 2024-04-27T05:20:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:(2/2): ceph-fuse-18.2.2-1258.g7d2137f0.el8.x86_ 1.9 MB/s | 849 kB 00:00 2024-04-27T05:20:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:01.298 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.3 MB/s | 929 kB 00:00 2024-04-27T05:20:01.309 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:01.331 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:01.331 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:01.409 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:01.409 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:01.599 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:01.738 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-27T05:20:01.803 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T05:20:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T05:20:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/2 2024-04-27T05:20:02.699 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-27T05:20:02.699 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:02.699 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:02.699 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-27T05:20:02.699 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:02.700 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:02.985 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-volume 2024-04-27T05:20:03.504 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:26 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:03.626 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 264 k 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-27T05:20:03.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout:Install 2 Packages 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 341 k 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 1.5 M 2024-04-27T05:20:03.628 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:04.003 INFO:teuthology.orchestra.run.smithi114.stdout:(1/2): ceph-volume-18.2.2-1258.g7d2137f0.el8.no 1.1 MB/s | 264 kB 00:00 2024-04-27T05:20:04.453 INFO:teuthology.orchestra.run.smithi114.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 113 kB/s | 77 kB 00:00 2024-04-27T05:20:04.453 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:04.454 INFO:teuthology.orchestra.run.smithi114.stdout:Total 412 kB/s | 341 kB 00:00 2024-04-27T05:20:04.465 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:04.474 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:04.474 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:04.498 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:04.499 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:04.635 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:04.809 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-27T05:20:04.838 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T05:20:04.854 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T05:20:04.854 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:20:04.854 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-27T05:20:04.854 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:05.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 1/2 2024-04-27T05:20:05.902 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-27T05:20:05.902 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:05.902 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:05.902 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:20:05.902 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-27T05:20:05.903 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:05.903 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:06.225 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados-devel 2024-04-27T05:20:06.576 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 745 kB/s | 6.0 MB 00:08 2024-04-27T05:20:06.579 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:06.580 INFO:teuthology.orchestra.run.smithi138.stdout:Total 843 kB/s | 7.0 MB 00:08 2024-04-27T05:20:06.652 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:06.665 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:06.665 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:06.744 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:29 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:06.761 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:06.761 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:06.867 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:06.868 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:06.868 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-04-27T05:20:06.868 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:06.868 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:06.868 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 127 k 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 127 k 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 456 k 2024-04-27T05:20:06.869 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:07.092 INFO:teuthology.orchestra.run.smithi114.stdout:librados-devel-18.2.2-1258.g7d2137f0.el8.x86_64 568 kB/s | 127 kB 00:00 2024-04-27T05:20:07.093 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:07.093 INFO:teuthology.orchestra.run.smithi114.stdout:Total 564 kB/s | 127 kB 00:00 2024-04-27T05:20:07.093 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:07.098 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:07.098 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:07.160 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:07.160 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:07.234 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-27T05:20:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-27T05:20:07.310 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-27T05:20:07.916 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-27T05:20:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-27T05:20:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 6/6 2024-04-27T05:20:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:08.069 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:08.069 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:08.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs2 2024-04-27T05:20:08.811 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:31 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:08.833 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 6/6 2024-04-27T05:20:08.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 1/6 2024-04-27T05:20:08.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-27T05:20:08.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-27T05:20:08.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-27T05:20:08.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-27T05:20:08.889 INFO:teuthology.orchestra.run.smithi114.stdout:Package libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:08.931 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:08.932 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:08.932 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:09.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs-devel 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:09.085 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:09.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-04-27T05:20:09.517 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:32 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:09.635 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:09.636 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:09.636 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 31 k 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 31 k 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 139 k 2024-04-27T05:20:09.637 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:09.821 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:32 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:09.889 INFO:teuthology.orchestra.run.smithi114.stdout:libcephfs-devel-18.2.2-1258.g7d2137f0.el8.x86_6 123 kB/s | 31 kB 00:00 2024-04-27T05:20:09.890 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:09.890 INFO:teuthology.orchestra.run.smithi114.stdout:Total 123 kB/s | 31 kB 00:00 2024-04-27T05:20:09.890 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:09.893 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:09.893 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:09.896 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:09.896 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:09.941 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:09.942 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 849 k 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 929 k 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.7 M 2024-04-27T05:20:09.943 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:10.084 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:10.285 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 426 kB/s | 80 kB 00:00 2024-04-27T05:20:10.400 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:10.436 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-fuse-18.2.2-1258.g7d2137f0.el8.x86_ 2.4 MB/s | 849 kB 00:00 2024-04-27T05:20:10.436 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:10.437 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.8 MB/s | 929 kB 00:00 2024-04-27T05:20:10.448 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:10.470 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:10.470 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:10.547 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:10.547 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:10.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:10.595 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:10.595 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:10.595 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:10.595 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:10.596 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:10.731 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:10.766 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados2 2024-04-27T05:20:10.867 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-27T05:20:10.932 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T05:20:11.275 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:34 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:11.353 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:11.394 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:11.394 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:11.394 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:11.466 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librbd1 2024-04-27T05:20:11.657 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2/2 2024-04-27T05:20:11.657 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/2 2024-04-27T05:20:11.897 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-27T05:20:11.897 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:11.897 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:11.897 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-27T05:20:11.897 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:11.897 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:11.978 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:34 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:12.057 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:12.097 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:12.098 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:12.098 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:12.160 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rados 2024-04-27T05:20:12.175 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-volume 2024-04-27T05:20:12.664 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:35 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:12.698 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:35 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:12.742 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:12.782 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:12.783 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:12.783 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:12.824 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:18.2.2-1258.g7d2137f0.el8 ceph-noarch 264 k 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:12.826 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:12.826 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-27T05:20:12.826 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:12.826 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 341 k 2024-04-27T05:20:12.826 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 1.5 M 2024-04-27T05:20:12.826 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:12.842 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rgw 2024-04-27T05:20:13.182 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 362 kB/s | 77 kB 00:00 2024-04-27T05:20:13.207 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-volume-18.2.2-1258.g7d2137f0.el8.no 1.1 MB/s | 264 kB 00:00 2024-04-27T05:20:13.208 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:13.208 INFO:teuthology.orchestra.run.smithi138.stdout:Total 889 kB/s | 341 kB 00:00 2024-04-27T05:20:13.220 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:13.229 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:13.229 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:13.254 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:13.255 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:13.347 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:36 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:13.392 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:13.425 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:13.466 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:13.467 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:13.467 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:13.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-cephfs 2024-04-27T05:20:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-27T05:20:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T05:20:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2/2 2024-04-27T05:20:13.625 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:20:13.626 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-27T05:20:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:14.028 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:37 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:14.106 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:14.148 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:14.149 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:14.149 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:14.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rbd 2024-04-27T05:20:14.307 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 1/2 2024-04-27T05:20:14.624 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-27T05:20:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:14.625 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:18.2.2-1258.g7d2137f0.el8.noarch 2024-04-27T05:20:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-27T05:20:14.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:14.625 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:14.723 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:37 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:14.801 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:14.841 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:14.842 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-04-27T05:20:14.842 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:14.901 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-fuse 2024-04-27T05:20:14.950 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-04-27T05:20:15.407 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:38 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:15.470 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:38 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:15.527 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 87 k 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:15.528 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:20:15.529 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:15.529 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 87 k 2024-04-27T05:20:15.529 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 227 k 2024-04-27T05:20:15.529 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:15.592 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 127 k 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:15.593 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:20:15.594 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:15.594 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 127 k 2024-04-27T05:20:15.594 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 456 k 2024-04-27T05:20:15.594 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:15.742 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-fuse-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 406 kB/s | 87 kB 00:00 2024-04-27T05:20:15.743 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:15.743 INFO:teuthology.orchestra.run.smithi114.stdout:Total 403 kB/s | 87 kB 00:00 2024-04-27T05:20:15.743 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:15.750 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:15.750 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:15.811 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:15.811 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:15.905 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-18.2.2-1258.g7d2137f0.el8.x86_64 407 kB/s | 127 kB 00:00 2024-04-27T05:20:15.906 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:15.906 INFO:teuthology.orchestra.run.smithi138.stdout:Total 405 kB/s | 127 kB 00:00 2024-04-27T05:20:15.906 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:15.911 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:15.911 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:15.951 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:15.974 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:15.974 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:16.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:16.125 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:16.251 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:16.612 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:16.612 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:16.612 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:16.612 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:16.612 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:16.613 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:16.648 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:16.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-mirror 2024-04-27T05:20:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:16.890 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:16.890 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:17.063 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-04-27T05:20:17.380 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:40 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:17.499 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.0 M 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:17.500 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:20:17.501 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:17.501 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 3.0 M 2024-04-27T05:20:17.501 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 12 M 2024-04-27T05:20:17.501 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:17.580 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:40 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:17.660 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:17.700 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:17.701 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:17.701 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:17.794 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-04-27T05:20:17.863 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-mirror-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 8.3 MB/s | 3.0 MB 00:00 2024-04-27T05:20:17.863 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:17.864 INFO:teuthology.orchestra.run.smithi114.stdout:Total 8.2 MB/s | 3.0 MB 00:00 2024-04-27T05:20:17.864 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:17.873 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:17.873 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:17.951 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:17.951 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:18.305 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:41 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:18.364 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:18.424 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 31 k 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:18.425 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:18.426 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:20:18.426 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:18.426 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 31 k 2024-04-27T05:20:18.426 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 139 k 2024-04-27T05:20:18.426 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:18.439 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:18.463 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:18.463 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:20:18.463 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-27T05:20:18.463 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T05:20:18.463 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T05:20:18.463 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:18.666 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-18.2.2-1258.g7d2137f0.el8.x86_6 129 kB/s | 31 kB 00:00 2024-04-27T05:20:18.667 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:18.667 INFO:teuthology.orchestra.run.smithi138.stdout:Total 128 kB/s | 31 kB 00:00 2024-04-27T05:20:18.667 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:18.671 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:18.671 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:18.673 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:18.673 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:18.756 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:18.863 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:19.205 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:19.406 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:19.406 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:19.406 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:19.406 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:19.407 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:19.407 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:19.466 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:19.467 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:19.467 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:19.467 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:19.467 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:19.467 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:19.571 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-04-27T05:20:19.729 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-nbd 2024-04-27T05:20:20.082 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:20.160 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:20.201 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:20.201 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:20.201 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:20.243 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:43 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:20.261 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-04-27T05:20:20.367 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 172 k 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-27T05:20:20.368 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-04-27T05:20:20.369 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-04-27T05:20:20.369 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:20.369 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 172 k 2024-04-27T05:20:20.369 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 494 k 2024-04-27T05:20:20.369 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-27T05:20:20.600 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-nbd-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 745 kB/s | 172 kB 00:00 2024-04-27T05:20:20.601 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:20.601 INFO:teuthology.orchestra.run.smithi114.stdout:Total 740 kB/s | 172 kB 00:00 2024-04-27T05:20:20.601 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-27T05:20:20.608 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-27T05:20:20.609 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-27T05:20:20.671 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-27T05:20:20.672 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-27T05:20:20.776 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:20.820 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-27T05:20:20.855 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:20.895 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:20.896 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:20.896 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:20.929 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:20.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-04-27T05:20:21.275 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:21.457 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:44 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:21.496 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:21.497 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:21.497 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-27T05:20:21.497 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:21.497 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:21.497 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-27T05:20:21.536 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:21.576 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:21.577 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:21.578 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:21.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-04-27T05:20:21.719 DEBUG:teuthology.parallel:result is None 2024-04-27T05:20:22.145 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:45 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:22.224 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:22.264 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:22.265 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:22.265 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:22.322 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-04-27T05:20:22.830 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:45 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:22.909 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:22.950 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:22.951 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:22.951 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:23.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-04-27T05:20:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:46 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:23.600 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 is already installed. 2024-04-27T05:20:23.641 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:23.642 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-27T05:20:23.642 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:23.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-04-27T05:20:24.200 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:47 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:24.320 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:24.321 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T05:20:24.321 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:24.321 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 87 k 2024-04-27T05:20:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 87 k 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 227 k 2024-04-27T05:20:24.322 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:24.528 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 421 kB/s | 87 kB 00:00 2024-04-27T05:20:24.528 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:24.529 INFO:teuthology.orchestra.run.smithi138.stdout:Total 418 kB/s | 87 kB 00:00 2024-04-27T05:20:24.529 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:24.536 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:24.536 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:24.597 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:24.598 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:24.848 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:25.200 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:25.450 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:25.450 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:25.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-04-27T05:20:26.219 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:49 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:26.345 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 3.0 M 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.0 M 2024-04-27T05:20:26.346 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 12 M 2024-04-27T05:20:26.347 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:26.731 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 7.8 MB/s | 3.0 MB 00:00 2024-04-27T05:20:26.732 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:26.732 INFO:teuthology.orchestra.run.smithi138.stdout:Total 7.7 MB/s | 3.0 MB 00:00 2024-04-27T05:20:26.732 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:26.742 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:26.742 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:26.826 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:26.826 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:27.258 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:27.332 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:27.358 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:27.358 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T05:20:27.358 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-27T05:20:27.359 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T05:20:27.359 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T05:20:27.359 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:28.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:28.366 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:28.366 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:28.366 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:28.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-04-27T05:20:29.081 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:52 ago on Sat 27 Apr 2024 05:18:37 AM UTC. 2024-04-27T05:20:29.212 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:18.2.2-1258.g7d2137f0.el8 ceph 172 k 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:29.213 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 172 k 2024-04-27T05:20:29.214 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 494 k 2024-04-27T05:20:29.214 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-27T05:20:29.559 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-18.2.2-1258.g7d2137f0.el8.x86_64.rpm 497 kB/s | 172 kB 00:00 2024-04-27T05:20:29.560 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-27T05:20:29.560 INFO:teuthology.orchestra.run.smithi138.stdout:Total 495 kB/s | 172 kB 00:00 2024-04-27T05:20:29.560 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-27T05:20:29.568 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-27T05:20:29.568 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-27T05:20:29.632 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-27T05:20:29.632 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-27T05:20:29.787 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-27T05:20:29.896 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:30.262 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:30.474 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 1/1 2024-04-27T05:20:30.474 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:30.474 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-27T05:20:30.474 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:18.2.2-1258.g7d2137f0.el8.x86_64 2024-04-27T05:20:30.474 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:30.474 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-27T05:20:30.713 DEBUG:teuthology.parallel:result is None 2024-04-27T05:20:30.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T05:20:30.937 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T05:20:30.962 INFO:teuthology.orchestra.run.smithi114.stdout:18.2.2-1258.g7d2137f0.el8 2024-04-27T05:20:30.963 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1258.g7d2137f0.el8 2024-04-27T05:20:30.963 INFO:teuthology.task.install:The correct ceph version 18.2.2-1258.g7d2137f0 is installed. 2024-04-27T05:20:30.965 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d2137f0a666c92a9411c1af5c33e4968be54da6 2024-04-27T05:20:31.191 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T05:20:31.212 INFO:teuthology.orchestra.run.smithi138.stdout:18.2.2-1258.g7d2137f0.el8 2024-04-27T05:20:31.212 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1258.g7d2137f0.el8 2024-04-27T05:20:31.212 INFO:teuthology.task.install:The correct ceph version 18.2.2-1258.g7d2137f0 is installed. 2024-04-27T05:20:31.214 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-27T05:20:31.214 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:31.214 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T05:20:31.243 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:31.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T05:20:31.276 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-27T05:20:31.277 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:31.277 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T05:20:31.309 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T05:20:31.375 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:31.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T05:20:31.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T05:20:31.463 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-27T05:20:31.463 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:31.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T05:20:31.488 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T05:20:31.560 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:31.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T05:20:31.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T05:20:31.647 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-27T05:20:31.648 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:31.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T05:20:31.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T05:20:31.735 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:31.735 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T05:20:31.758 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T05:20:31.821 INFO:teuthology.run_tasks:Running task ceph... 2024-04-27T05:20:31.906 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-27T05:20:31.906 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-04-27T05:20:31.908 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-04-27T05:20:31.931 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-27T05:20:31.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T05:20:31.973 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T05:20:31.995 INFO:tasks.ceph:Creating extra log directories... 2024-04-27T05:20:31.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T05:20:32.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T05:20:32.069 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-27T05:20:32.069 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-27T05:20:32.069 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743', 'branch': 'reef', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7675743', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'wait-for-scrub': False}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6'}}, 'rgw': {'ec-data-pool': False, 'frontend': 'beast'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '5403a02127f27ec6c3309ef057522ab70cdeb9d2'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 4225, 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'rgw', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_5403a02127f27ec6c3309ef057522ab70cdeb9d2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '5403a02127f27ec6c3309ef057522ab70cdeb9d2', 'targets': {'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIIIWIdHm4s3w36bsFdI0PrVydeWjs+kcw3nwWuwgFCTc0Jp8ALO2Npj8/i2DmSI1SNzXOX3XylubebHIIPmJ5E=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmHQcWGcmLsqTmo2hP/NZlhARlo3SnPTiQJKMcysv2CPO/7aeC4hJlBMWWpG8L+Wr70LQjwbl+zuLv7JQVIT6A='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '7d2137f0a666c92a9411c1af5c33e4968be54da6', 'wait-for-scrub': False, 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_22:40:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-27T05:20:32.070 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T05:20:32.102 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T05:20:32.116 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T05:20:32.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T05:20:32.180 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:32.180 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T05:20:32.233 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T05:20:32.233 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-04-27T05:20:32.287 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T05:20:32.287 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:32.287 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 642 Links: 1 2024-04-27T05:20:32.287 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:32.288 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:32.288 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-27 05:20:18.717747932 +0000 2024-04-27T05:20:32.288 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-27 05:18:00.291143081 +0000 2024-04-27T05:20:32.288 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-27 05:18:00.291143081 +0000 2024-04-27T05:20:32.288 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-27 05:18:00.291143081 +0000 2024-04-27T05:20:32.289 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T05:20:32.349 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-27T05:20:32.349 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-27T05:20:32.349 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000178861 s, 2.9 MB/s 2024-04-27T05:20:32.350 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T05:20:32.406 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-27 05:20:18.717747932 +0000 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-27 05:18:00.614135713 +0000 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-27 05:18:00.614135713 +0000 2024-04-27T05:20:32.460 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-27 05:18:00.614135713 +0000 2024-04-27T05:20:32.461 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T05:20:32.521 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-27T05:20:32.521 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-27T05:20:32.521 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000232667 s, 2.2 MB/s 2024-04-27T05:20:32.522 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T05:20:32.578 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-27 05:20:18.718747911 +0000 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-27 05:18:00.957127888 +0000 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-27 05:18:00.957127888 +0000 2024-04-27T05:20:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-27 05:18:00.957127888 +0000 2024-04-27T05:20:32.633 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T05:20:32.694 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-27T05:20:32.694 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-27T05:20:32.694 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000227336 s, 2.3 MB/s 2024-04-27T05:20:32.695 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T05:20:32.751 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-04-27T05:20:32.806 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T05:20:32.806 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:32.806 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 697 Links: 1 2024-04-27T05:20:32.807 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:32.807 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:32.807 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-27 05:20:18.718747911 +0000 2024-04-27T05:20:32.807 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-27 05:18:01.309119859 +0000 2024-04-27T05:20:32.807 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-27 05:18:01.309119859 +0000 2024-04-27T05:20:32.807 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-27 05:18:01.309119859 +0000 2024-04-27T05:20:32.807 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T05:20:32.871 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-27T05:20:32.871 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-27T05:20:32.871 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000172651 s, 3.0 MB/s 2024-04-27T05:20:32.871 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T05:20:32.926 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-27T05:20:32.926 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:32.926 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T05:20:32.943 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T05:20:32.943 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-27T05:20:33.000 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T05:20:33.000 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:33.000 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-27T05:20:33.000 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:33.001 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:33.001 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 05:20:27.595333653 +0000 2024-04-27T05:20:33.001 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 05:18:00.292763171 +0000 2024-04-27T05:20:33.001 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 05:18:00.292763171 +0000 2024-04-27T05:20:33.001 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-27 05:18:00.292763171 +0000 2024-04-27T05:20:33.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T05:20:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T05:20:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T05:20:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000212738 s, 2.4 MB/s 2024-04-27T05:20:33.066 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T05:20:33.123 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 05:20:27.595333653 +0000 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 05:18:00.638755377 +0000 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 05:18:00.638755377 +0000 2024-04-27T05:20:33.179 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-27 05:18:00.638755377 +0000 2024-04-27T05:20:33.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T05:20:33.242 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T05:20:33.243 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T05:20:33.243 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000216998 s, 2.4 MB/s 2024-04-27T05:20:33.244 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T05:20:33.300 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-27T05:20:33.355 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T05:20:33.355 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:33.355 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-27T05:20:33.355 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:33.355 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:33.355 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 05:20:27.595333653 +0000 2024-04-27T05:20:33.356 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 05:18:00.976747763 +0000 2024-04-27T05:20:33.356 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 05:18:00.976747763 +0000 2024-04-27T05:20:33.356 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-27 05:18:00.976747763 +0000 2024-04-27T05:20:33.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T05:20:33.418 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T05:20:33.418 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T05:20:33.418 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000211158 s, 2.4 MB/s 2024-04-27T05:20:33.419 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T05:20:33.475 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-27T05:20:33.531 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T05:20:33.531 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T05:20:33.531 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-27T05:20:33.531 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T05:20:33.532 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T05:20:33.532 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-27 05:20:27.596333632 +0000 2024-04-27T05:20:33.532 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-27 05:18:01.312740193 +0000 2024-04-27T05:20:33.532 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-27 05:18:01.312740193 +0000 2024-04-27T05:20:33.532 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-27 05:18:01.312740193 +0000 2024-04-27T05:20:33.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T05:20:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-27T05:20:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-27T05:20:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000182644 s, 2.8 MB/s 2024-04-27T05:20:33.594 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T05:20:33.651 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T05:20:33.651 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-27T05:20:33.651 INFO:tasks.ceph:Generating config... 2024-04-27T05:20:33.652 INFO:tasks.ceph:[client] debug ms = 1 2024-04-27T05:20:33.652 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-04-27T05:20:33.653 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-27T05:20:33.653 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-27T05:20:33.653 INFO:tasks.ceph:[client] setuser = ceph 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] debug objclass = 20 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-27T05:20:33.653 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-27T05:20:33.654 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-27T05:20:33.654 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-27T05:20:33.654 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-27T05:20:33.654 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-27T05:20:33.654 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-27T05:20:33.654 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-27T05:20:33.654 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-27T05:20:33.654 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-27T05:20:33.654 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-27T05:20:33.654 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-27T05:20:33.654 INFO:tasks.ceph:Setting up mon.a... 2024-04-27T05:20:33.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-27T05:20:33.700 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.keyring 2024-04-27T05:20:33.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-27T05:20:33.742 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T05:20:33.805 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.114')] 2024-04-27T05:20:33.806 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.114'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-04-27T05:20:33.807 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:33.807 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T05:20:33.860 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.114 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:33.938 INFO:teuthology.orchestra.run.smithi114.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:20:33.938 INFO:teuthology.orchestra.run.smithi114.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: generated fsid 5659e27f-2999-4d49-8a80-9d85e921a02f 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = pacific 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 0 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:fsid 5659e27f-2999-4d49-8a80-9d85e921a02f 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:last_changed 2024-04-27T05:20:33.938251+0000 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-04-27T05:20:33.938251+0000 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:min_mon_release 16 (pacific) 2024-04-27T05:20:33.939 INFO:teuthology.orchestra.run.smithi114.stdout:election_strategy: 1 2024-04-27T05:20:33.940 INFO:teuthology.orchestra.run.smithi114.stdout:0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-04-27T05:20:33.940 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-27T05:20:33.941 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T05:20:33.995 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5659e27f-2999-4d49-8a80-9d85e921a02f... 2024-04-27T05:20:33.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T05:20:34.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.075 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.076 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.077 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = 172.21.15.114 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 5659e27f-2999-4d49-8a80-9d85e921a02f 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-04-27T05:20:34.078 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.079 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: debug objclass = 20 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: bdev async discard = True 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: bdev enable discard = True 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore allocator = bitmap 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-04-27T05:20:34.080 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 1/20 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 1/20 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 4/10 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-04-27T05:20:34.081 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-04-27T05:20:34.082 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-04-27T05:20:34.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: rgw gc obj min wait = 15 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: debug rgw = 20 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: setgroup = ceph 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout: setuser = ceph 2024-04-27T05:20:34.084 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.085 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-04-27T05:20:34.086 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.087 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = 172.21.15.114 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 5659e27f-2999-4d49-8a80-9d85e921a02f 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-04-27T05:20:34.088 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.089 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: debug objclass = 20 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: bdev async discard = True 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: bdev enable discard = True 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore allocator = bitmap 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-04-27T05:20:34.090 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 1/20 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 1/20 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 4/10 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-27T05:20:34.091 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-04-27T05:20:34.092 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:20:34.093 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: rgw gc obj min wait = 15 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: debug rgw = 20 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: setgroup = ceph 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout: setuser = ceph 2024-04-27T05:20:34.094 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-04-27T05:20:34.096 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-27T05:20:34.096 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-27T05:20:34.187 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-27T05:20:34.187 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:34.187 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-27T05:20:34.201 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:34.202 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-27T05:20:34.256 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-04-27T05:20:34.256 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:34.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T05:20:34.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T05:20:34.329 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:34.329 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:34.383 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:20:34.383 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:34.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T05:20:34.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T05:20:34.417 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:34.418 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:34.472 INFO:tasks.ceph:Setting up mon nodes... 2024-04-27T05:20:34.473 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-27T05:20:34.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T05:20:34.567 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T05:20:34.569 INFO:tasks.ceph:Setting up mds nodes... 2024-04-27T05:20:34.569 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-27T05:20:34.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-27T05:20:34.608 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-27T05:20:34.619 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-27T05:20:34.620 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-27T05:20:34.620 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-27T05:20:34.642 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-27T05:20:34.643 INFO:tasks.ceph:role: osd.0 2024-04-27T05:20:34.643 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-27T05:20:34.643 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T05:20:34.714 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T05:20:35.211 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-04-27T05:20:35.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-04-27T05:20:35.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-27T05:20:35.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-27T05:20:35.370 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:35.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:35.456+0000 7f698101d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-27T05:20:35.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:35.460+0000 7f698101d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-27T05:20:35.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:35.460+0000 7f698101d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T05:20:35.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:35.460+0000 7f698101d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T05:20:35.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:35.460+0000 7f698101d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T05:20:35.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:35.465+0000 7f698101d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-27T05:20:38.310 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T05:20:38.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-27T05:20:38.363 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T05:20:38.363 INFO:tasks.ceph:role: osd.1 2024-04-27T05:20:38.363 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:20:38.363 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T05:20:38.439 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T05:20:38.440 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T05:20:38.936 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-27T05:20:38.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-27T05:20:38.938 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-27T05:20:39.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-27T05:20:39.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-27T05:20:39.166 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T05:20:39.166 INFO:tasks.ceph:role: osd.2 2024-04-27T05:20:39.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:20:39.166 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T05:20:39.239 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T05:20:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T05:20:39.240 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T05:20:39.695 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-27T05:20:39.698 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-27T05:20:39.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-04-27T05:20:39.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-27T05:20:39.844 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-27T05:20:39.910 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T05:20:39.910 INFO:tasks.ceph:role: osd.3 2024-04-27T05:20:39.910 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:20:39.910 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T05:20:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T05:20:39.981 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T05:20:39.981 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T05:20:39.981 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T05:20:39.982 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T05:20:39.982 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-27T05:20:39.982 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T05:20:39.982 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T05:20:39.982 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T05:20:39.982 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T05:20:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-27T05:20:40.472 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-27T05:20:40.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-04-27T05:20:40.590 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-27T05:20:40.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:40.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:40.709+0000 7f6cc9c11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-27T05:20:40.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:40.711+0000 7f6cc9c11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-27T05:20:40.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:40.711+0000 7f6cc9c11640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T05:20:40.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:40.711+0000 7f6cc9c11640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T05:20:40.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:40.711+0000 7f6cc9c11640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T05:20:40.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:40.718+0000 7f6cc9c11640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-27T05:20:43.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T05:20:43.594 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:43.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:43.674+0000 7f9992587640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-27T05:20:43.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:43.674+0000 7f9992587640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-27T05:20:43.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:43.674+0000 7f9992587640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T05:20:43.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:43.674+0000 7f9992587640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T05:20:43.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:43.674+0000 7f9992587640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T05:20:43.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:43.675+0000 7f9992587640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-27T05:20:46.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T05:20:46.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:46.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:46.675+0000 7f3197a1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-27T05:20:46.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:46.675+0000 7f3197a1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-27T05:20:46.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:46.675+0000 7f3197a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T05:20:46.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:46.675+0000 7f3197a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T05:20:46.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:46.675+0000 7f3197a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T05:20:46.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:46.676+0000 7f3197a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-27T05:20:49.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T05:20:49.560 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-27T05:20:49.560 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:49.560 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-27T05:20:49.589 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:49.589 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-27T05:20:49.626 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:49.626 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-27T05:20:49.691 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:49.692 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-27T05:20:49.756 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:49.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-27T05:20:49.822 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:49.822 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-27T05:20:49.879 INFO:tasks.ceph:Adding keys to all mons... 2024-04-27T05:20:49.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAjiyxm7AtIGxAApxu80uHYAsgdE7UpW4HFCA== 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAiiyxmBHnCIRAA8iRTBKQqZKayUgYXGTPSnw== 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAoiyxm7P9QKhAArGvUrSzkyLpFYBfcBpnR4A== 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-04-27T05:20:49.906 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAriyxmZQxKKBAAjVLuOcXwkjrx3v4PMdRD5w== 2024-04-27T05:20:49.907 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2024-04-27T05:20:49.907 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAuiyxmwd1XKBAACgN2T6nPSrEwceTiDFyflg== 2024-04-27T05:20:49.907 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-04-27T05:20:49.907 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAiiyxmcF80JBAAVa1mRTxogp+y38yk255c4Q== 2024-04-27T05:20:49.908 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T05:20:50.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T05:20:50.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T05:20:50.105 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T05:20:50.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T05:20:50.201 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T05:20:50.252 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-27T05:20:50.252 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-27T05:20:50.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T05:20:50.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-27T05:20:50.391 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T05:20:50.446 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-27T05:20:50.446 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-27T05:20:50.447 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-27T05:20:50.490 INFO:tasks.ceph.mon.a:Started 2024-04-27T05:20:50.490 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-27T05:20:50.490 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-27T05:20:50.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-27T05:20:50.492 INFO:tasks.ceph.mgr.x:Started 2024-04-27T05:20:50.493 INFO:tasks.ceph:Setting crush tunables to default 2024-04-27T05:20:50.493 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd crush tunables default 2024-04-27T05:20:50.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.586+0000 7fd7c9db9640 1 Processor -- start 2024-04-27T05:20:50.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.586+0000 7fd7c9db9640 1 -- start start 2024-04-27T05:20:50.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c9db9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c9db9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c40996b0 con 0x7fd7c4098240 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c9db9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c40964a0 con 0x7fd7c4097920 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59776/0 (socket says 172.21.15.114:59776) 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 -- 172.21.15.114:0/507543519 learned_addr learned my addr 172.21.15.114:0/507543519 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c2ffd640 1 --1- 172.21.15.114:0/507543519 >> v1:172.21.15.114:6789/0 conn(0x7fd7c4098240 0x7fd7c4131580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:49940/0 (socket says 172.21.15.114:49940) 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 -- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 msgr2=0x7fd7c4097d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 -- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 msgr2=0x7fd7c4097d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 --2- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T05:20:50.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.587+0000 7fd7c37fe640 1 --2- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T05:20:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.598+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927503760 0 0) 0x7fd7c40996b0 con 0x7fd7c4098240 2024-04-27T05:20:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.598+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b0003520 con 0x7fd7c4098240 2024-04-27T05:20:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1317038579 0 0) 0x7fd7b0003520 con 0x7fd7c4098240 2024-04-27T05:20:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 msgr2=0x7fd7c4097d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T05:20:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 --2- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4098a30 con 0x7fd7c4098240 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 203229478 0 0) 0x7fd7ac0015a0 con 0x7fd7c4098240 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 >> v1:172.21.15.114:6789/0 conn(0x7fd7c4098240 legacy=0x7fd7c4131580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 --2- 172.21.15.114:0/507543519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7b0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40964a0 con 0x7fd7b0003970 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c37fe640 1 --2- 172.21.15.114:0/507543519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7b0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.599+0000 7fd7c37fe640 1 -- 172.21.15.114:0/507543519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0003520 con 0x7fd7b0003970 2024-04-27T05:20:50.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c37fe640 1 --2- 172.21.15.114:0/507543519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7b0003d70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009c40 tx=0x7fd7b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3b7b9d05736ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:50.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b402cbd0 con 0x7fd7b0003970 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4030070 con 0x7fd7b0003970 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c27fc640 1 -- 172.21.15.114:0/507543519 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b402cea0 con 0x7fd7b0003970 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 -- 172.21.15.114:0/507543519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 msgr2=0x7fd7b0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 --2- 172.21.15.114:0/507543519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7b0003d70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009c40 tx=0x7fd7b00057f0 comp rx=0 tx=0).stop 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 -- 172.21.15.114:0/507543519 shutdown_connections 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 --2- 172.21.15.114:0/507543519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7b0003d70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 --2- 172.21.15.114:0/507543519 >> v2:172.21.15.114:3300/0 conn(0x7fd7c4097920 0x7fd7c4097d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 -- 172.21.15.114:0/507543519 >> 172.21.15.114:0/507543519 conn(0x7fd7c409d940 msgr2=0x7fd7c409dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 -- 172.21.15.114:0/507543519 shutdown_connections 2024-04-27T05:20:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.601+0000 7fd7c9db9640 1 -- 172.21.15.114:0/507543519 wait complete. 2024-04-27T05:20:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.602+0000 7fd7c9db9640 1 Processor -- start 2024-04-27T05:20:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c9db9640 1 -- start start 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c9db9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7c41c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c9db9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4095170 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7c41c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c37fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7c41c12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59780/0 (socket says 172.21.15.114:59780) 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c37fe640 1 -- 172.21.15.114:0/3630933838 learned_addr learned my addr 172.21.15.114:0/3630933838 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c37fe640 1 -- 172.21.15.114:0/3630933838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c41a9310 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c37fe640 1 --2- 172.21.15.114:0/3630933838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7c41c12c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd7b402c6c0 tx=0x7fd7b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4036e30 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b4040400 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c4095380 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.603+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b40406c0 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.604+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c4094540 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.604+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd7b4040820 con 0x7fd7b0003970 2024-04-27T05:20:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.604+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd7b4049950 con 0x7fd7b0003970 2024-04-27T05:20:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.604+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c4061a20 con 0x7fd7b0003970 2024-04-27T05:20:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.607+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd7b4044030 con 0x7fd7b0003970 2024-04-27T05:20:50.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.668+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd7c4070d80 con 0x7fd7b0003970 2024-04-27T05:20:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.670+0000 7fd7bbfff640 1 -- 172.21.15.114:0/3630933838 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd7b4032070 con 0x7fd7b0003970 2024-04-27T05:20:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2024-04-27T05:20:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 msgr2=0x7fd7c41c12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 --2- 172.21.15.114:0/3630933838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7c41c12c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd7b402c6c0 tx=0x7fd7b4004a40 comp rx=0 tx=0).stop 2024-04-27T05:20:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 shutdown_connections 2024-04-27T05:20:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 --2- 172.21.15.114:0/3630933838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7b0003970 0x7fd7c41c12c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 >> 172.21.15.114:0/3630933838 conn(0x7fd7c409d940 msgr2=0x7fd7c406ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 shutdown_connections 2024-04-27T05:20:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:50.671+0000 7fd7c9db9640 1 -- 172.21.15.114:0/3630933838 wait complete. 2024-04-27T05:20:50.683 INFO:tasks.ceph:check_enable_crimson: False 2024-04-27T05:20:50.684 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-27T05:20:50.684 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-27T05:20:50.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-27T05:20:50.707 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:50.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-27T05:20:50.730 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:50.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-27T05:20:50.795 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-27T05:20:50.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-27T05:20:50.860 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 9b51ec11-b8cb-42da-a911-1aa825f7a690 0 2024-04-27T05:20:50.870 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:50.867+0000 7f070da0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.964+0000 7f9c09986640 1 Processor -- start 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.964+0000 7f9c09986640 1 -- start start 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.964+0000 7f9c09986640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9c041172c0 0x7f9c041196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c09986640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c04119bf0 con 0x7f9c04114560 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c09986640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0411ad90 con 0x7f9c041172c0 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c027fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9c041172c0 0x7f9c041196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:50.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c027fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9c041172c0 0x7f9c041196b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40500/0 (socket says 172.21.15.138:40500) 2024-04-27T05:20:50.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c027fc640 1 -- 172.21.15.138:0/4221489320 learned_addr learned my addr 172.21.15.138:0/4221489320 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:50.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c02ffd640 1 --1- 172.21.15.138:0/4221489320 >> v1:172.21.15.114:6789/0 conn(0x7f9c04114560 0x7f9c04114940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:59866/0 (socket says 172.21.15.138:59866) 2024-04-27T05:20:50.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c01ffb640 1 -- 172.21.15.138:0/4221489320 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586399331 0 0) 0x7f9c04119bf0 con 0x7f9c04114560 2024-04-27T05:20:50.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c01ffb640 1 -- 172.21.15.138:0/4221489320 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0003520 con 0x7f9c04114560 2024-04-27T05:20:50.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c027fc640 1 -- 172.21.15.138:0/4221489320 >> v1:172.21.15.114:6789/0 conn(0x7f9c04114560 legacy=0x7f9c04114940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:50.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.965+0000 7f9c027fc640 1 -- 172.21.15.138:0/4221489320 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0411aed0 con 0x7f9c041172c0 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.966+0000 7f9c027fc640 1 --2- 172.21.15.138:0/4221489320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c041172c0 0x7f9c041196b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097f0 tx=0x7f9bf402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e625178e01ca3741 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.966+0000 7f9c01ffb640 1 -- 172.21.15.138:0/4221489320 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bf40317a0 con 0x7f9c041172c0 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.966+0000 7f9c01ffb640 1 -- 172.21.15.138:0/4221489320 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf4031900 con 0x7f9c041172c0 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.966+0000 7f9c01ffb640 1 -- 172.21.15.138:0/4221489320 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bf4031bc0 con 0x7f9c041172c0 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 -- 172.21.15.138:0/4221489320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c041172c0 msgr2=0x7f9c041196b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 --2- 172.21.15.138:0/4221489320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c041172c0 0x7f9c041196b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097f0 tx=0x7f9bf402b910 comp rx=0 tx=0).stop 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 -- 172.21.15.138:0/4221489320 shutdown_connections 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 --2- 172.21.15.138:0/4221489320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c041172c0 0x7f9c041196b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 -- 172.21.15.138:0/4221489320 >> 172.21.15.138:0/4221489320 conn(0x7f9c0406d9d0 msgr2=0x7f9c0406ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:50.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 -- 172.21.15.138:0/4221489320 shutdown_connections 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 -- 172.21.15.138:0/4221489320 wait complete. 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.967+0000 7f9c09986640 1 Processor -- start 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c09986640 1 -- start start 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c09986640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 0x7f9c0410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c09986640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0410e480 con 0x7f9c04114560 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 0x7f9c0410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c02ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 0x7f9c0410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40506/0 (socket says 172.21.15.138:40506) 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c02ffd640 1 -- 172.21.15.138:0/960809737 learned_addr learned my addr 172.21.15.138:0/960809737 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.968+0000 7f9c02ffd640 1 -- 172.21.15.138:0/960809737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40094d0 con 0x7f9c04114560 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9c02ffd640 1 --2- 172.21.15.138:0/960809737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 0x7f9c0410df40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c410 tx=0x7f9bec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bec0074d0 con 0x7f9c04114560 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec007630 con 0x7f9c04114560 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0410e620 con 0x7f9c04114560 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bec004950 con 0x7f9c04114560 2024-04-27T05:20:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0410eab0 con 0x7f9c04114560 2024-04-27T05:20:50.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9bec01c450 con 0x7f9c04114560 2024-04-27T05:20:50.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.969+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9bec012770 con 0x7f9c04114560 2024-04-27T05:20:50.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.970+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0404e9b0 con 0x7f9c04114560 2024-04-27T05:20:50.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:50.972+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f9bec0129d0 con 0x7f9c04114560 2024-04-27T05:20:51.032 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:51.029+0000 7f070da0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T05:20:51.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.044+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9b51ec11-b8cb-42da-a911-1aa825f7a690", "id": 0} v 0) v1 -- 0x7f9c0410edb0 con 0x7f9c04114560 2024-04-27T05:20:51.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.046+0000 7f9bfbfff640 1 -- 172.21.15.138:0/960809737 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b51ec11-b8cb-42da-a911-1aa825f7a690", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9bec012ca0 con 0x7f9c04114560 2024-04-27T05:20:51.049 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 msgr2=0x7f9c0410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 --2- 172.21.15.138:0/960809737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 0x7f9c0410df40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c410 tx=0x7f9bec00c8d0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 shutdown_connections 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 --2- 172.21.15.138:0/960809737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04114560 0x7f9c0410df40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 >> 172.21.15.138:0/960809737 conn(0x7f9c0406d9d0 msgr2=0x7f9c041176a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 shutdown_connections 2024-04-27T05:20:51.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.047+0000 7f9c09986640 1 -- 172.21.15.138:0/960809737 wait complete. 2024-04-27T05:20:51.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 2b2e00dd-46de-4ed6-8ea8-cd251b08e55d 1 2024-04-27T05:20:51.088 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:51.085+0000 7f070da0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T05:20:51.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.152+0000 7febf441c640 1 Processor -- start 2024-04-27T05:20:51.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf441c640 1 -- start start 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf441c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7febec110950 0x7febec110d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf441c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febec111330 con 0x7febec1152d0 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf441c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febec115e70 con 0x7febec110950 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf21b8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7febec110950 0x7febec110d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf21b8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7febec110950 0x7febec110d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40508/0 (socket says 172.21.15.138:40508) 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf21b8640 1 -- 172.21.15.138:0/1100753423 learned_addr learned my addr 172.21.15.138:0/1100753423 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.153+0000 7febf19b7640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7febec1152d0 0x7febec115760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:59878/0 (socket says 172.21.15.138:59878) 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.154+0000 7febf21b8640 1 -- 172.21.15.138:0/1100753423 >> v1:172.21.15.114:6789/0 conn(0x7febec1152d0 legacy=0x7febec115760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.154+0000 7febf21b8640 1 -- 172.21.15.138:0/1100753423 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec115fe0 con 0x7febec110950 2024-04-27T05:20:51.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.154+0000 7febf11b6640 1 -- 172.21.15.138:0/1100753423 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126326717 0 0) 0x7febec111330 con 0x7febec1152d0 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.154+0000 7febf21b8640 1 --2- 172.21.15.138:0/1100753423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec110d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febe80097f0 tx=0x7febe802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf30a229c71eb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.154+0000 7febf11b6640 1 -- 172.21.15.138:0/1100753423 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febe80317a0 con 0x7febec110950 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf11b6640 1 -- 172.21.15.138:0/1100753423 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8031900 con 0x7febec110950 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf11b6640 1 -- 172.21.15.138:0/1100753423 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febe8031bc0 con 0x7febec110950 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 -- 172.21.15.138:0/1100753423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 msgr2=0x7febec110d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 --2- 172.21.15.138:0/1100753423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec110d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febe80097f0 tx=0x7febe802b910 comp rx=0 tx=0).stop 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 -- 172.21.15.138:0/1100753423 shutdown_connections 2024-04-27T05:20:51.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 --2- 172.21.15.138:0/1100753423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec110d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 -- 172.21.15.138:0/1100753423 >> 172.21.15.138:0/1100753423 conn(0x7febec06c190 msgr2=0x7febec06c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 -- 172.21.15.138:0/1100753423 shutdown_connections 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.155+0000 7febf441c640 1 -- 172.21.15.138:0/1100753423 wait complete. 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf441c640 1 Processor -- start 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf441c640 1 -- start start 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf441c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf441c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec1a52e0 con 0x7febec110950 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf21b8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf21b8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec1a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40514/0 (socket says 172.21.15.138:40514) 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.156+0000 7febf21b8640 1 -- 172.21.15.138:0/101136844 learned_addr learned my addr 172.21.15.138:0/101136844 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febf21b8640 1 -- 172.21.15.138:0/101136844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80094d0 con 0x7febec110950 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febf21b8640 1 --2- 172.21.15.138:0/101136844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec1a4d10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febe80097f0 tx=0x7febe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febe803ac00 con 0x7febec110950 2024-04-27T05:20:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe803ad60 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febe80443f0 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec1a54e0 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec1a9400 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7febe80448a0 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.157+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7febe802d040 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.158+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febec04e9b0 con 0x7febec110950 2024-04-27T05:20:51.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.160+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7febe8035020 con 0x7febec110950 2024-04-27T05:20:51.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.232+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2b2e00dd-46de-4ed6-8ea8-cd251b08e55d", "id": 1} v 0) v1 -- 0x7febec1152d0 con 0x7febec110950 2024-04-27T05:20:51.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.233+0000 7febe2ffd640 1 -- 172.21.15.138:0/101136844 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b2e00dd-46de-4ed6-8ea8-cd251b08e55d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7febe804d450 con 0x7febec110950 2024-04-27T05:20:51.236 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 msgr2=0x7febec1a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 --2- 172.21.15.138:0/101136844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec1a4d10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febe80097f0 tx=0x7febe80046c0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 shutdown_connections 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 --2- 172.21.15.138:0/101136844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febec110950 0x7febec1a4d10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 >> 172.21.15.138:0/101136844 conn(0x7febec06c190 msgr2=0x7febec112440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 shutdown_connections 2024-04-27T05:20:51.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.234+0000 7febf441c640 1 -- 172.21.15.138:0/101136844 wait complete. 2024-04-27T05:20:51.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new b957c5fa-4cab-40a2-bf4d-e6fe710c7d27 2 2024-04-27T05:20:51.337 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:51.335+0000 7f070da0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T05:20:51.385 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:51.382+0000 7f070da0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.382+0000 7fd654a10640 1 Processor -- start 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd654a10640 1 -- start start 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd654a10640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd650113560 0x7fd650113940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd654a10640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd650113fd0 con 0x7fd65010d330 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd654a10640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65010def0 con 0x7fd650113560 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd64e59c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd650113560 0x7fd650113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd64e59c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd650113560 0x7fd650113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40520/0 (socket says 172.21.15.138:40520) 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd64dd9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd65010d330 0x7fd65010d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:59884/0 (socket says 172.21.15.138:59884) 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.383+0000 7fd64e59c640 1 -- 172.21.15.138:0/945237180 learned_addr learned my addr 172.21.15.138:0/945237180 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64e59c640 1 -- 172.21.15.138:0/945237180 >> v1:172.21.15.114:6789/0 conn(0x7fd65010d330 legacy=0x7fd65010d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64e59c640 1 -- 172.21.15.138:0/945237180 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65010e060 con 0x7fd650113560 2024-04-27T05:20:51.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64d59a640 1 -- 172.21.15.138:0/945237180 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482706150 0 0) 0x7fd650113fd0 con 0x7fd65010d330 2024-04-27T05:20:51.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64e59c640 1 --2- 172.21.15.138:0/945237180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd650113560 0x7fd650113940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd6400097f0 tx=0x7fd64002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56972fe665746ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64d59a640 1 -- 172.21.15.138:0/945237180 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6400317a0 con 0x7fd650113560 2024-04-27T05:20:51.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64d59a640 1 -- 172.21.15.138:0/945237180 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd640031900 con 0x7fd650113560 2024-04-27T05:20:51.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.384+0000 7fd64d59a640 1 -- 172.21.15.138:0/945237180 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd640031bc0 con 0x7fd650113560 2024-04-27T05:20:51.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 -- 172.21.15.138:0/945237180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd650113560 msgr2=0x7fd650113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 --2- 172.21.15.138:0/945237180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd650113560 0x7fd650113940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd6400097f0 tx=0x7fd64002b910 comp rx=0 tx=0).stop 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 -- 172.21.15.138:0/945237180 shutdown_connections 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 --2- 172.21.15.138:0/945237180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd650113560 0x7fd650113940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 -- 172.21.15.138:0/945237180 >> 172.21.15.138:0/945237180 conn(0x7fd65006b340 msgr2=0x7fd65006d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 -- 172.21.15.138:0/945237180 shutdown_connections 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.385+0000 7fd654a10640 1 -- 172.21.15.138:0/945237180 wait complete. 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.386+0000 7fd654a10640 1 Processor -- start 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.386+0000 7fd654a10640 1 -- start start 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.386+0000 7fd654a10640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 0x7fd6501a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.386+0000 7fd654a10640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6501a3d60 con 0x7fd65010d330 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd64e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 0x7fd6501a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd64e59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 0x7fd6501a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40532/0 (socket says 172.21.15.138:40532) 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd64e59c640 1 -- 172.21.15.138:0/3458618595 learned_addr learned my addr 172.21.15.138:0/3458618595 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:51.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd64e59c640 1 -- 172.21.15.138:0/3458618595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6400094d0 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd64e59c640 1 --2- 172.21.15.138:0/3458618595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 0x7fd6501a3790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd640000c00 tx=0x7fd6400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd64003abe0 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64003ad40 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6501a3f00 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6400443f0 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6501a4350 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd640044850 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.387+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd64002d040 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.388+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6501a45f0 con 0x7fd65010d330 2024-04-27T05:20:51.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.390+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd640035020 con 0x7fd65010d330 2024-04-27T05:20:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.460+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b957c5fa-4cab-40a2-bf4d-e6fe710c7d27", "id": 2} v 0) v1 -- 0x7fd65004e9b0 con 0x7fd65010d330 2024-04-27T05:20:51.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.462+0000 7fd63effd640 1 -- 172.21.15.138:0/3458618595 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b957c5fa-4cab-40a2-bf4d-e6fe710c7d27", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd64004d580 con 0x7fd65010d330 2024-04-27T05:20:51.464 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-04-27T05:20:51.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 msgr2=0x7fd6501a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 --2- 172.21.15.138:0/3458618595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 0x7fd6501a3790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd640000c00 tx=0x7fd6400046c0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 shutdown_connections 2024-04-27T05:20:51.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 --2- 172.21.15.138:0/3458618595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd65010d330 0x7fd6501a3790 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 >> 172.21.15.138:0/3458618595 conn(0x7fd65006b340 msgr2=0x7fd65010f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 shutdown_connections 2024-04-27T05:20:51.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.463+0000 7fd654a10640 1 -- 172.21.15.138:0/3458618595 wait complete. 2024-04-27T05:20:51.476 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 224d0382-aff8-4501-95a0-060872ad7f05 3 2024-04-27T05:20:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.569+0000 7f235e175640 1 Processor -- start 2024-04-27T05:20:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.569+0000 7f235e175640 1 -- start start 2024-04-27T05:20:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f235e175640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2358117280 0x7f2358119670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f235e175640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2358119bb0 con 0x7f2358114490 2024-04-27T05:20:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f235e175640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235811ad50 con 0x7f2358117280 2024-04-27T05:20:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f234ffff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2358117280 0x7f2358119670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f23577fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2358114490 0x7f2358114870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:59886/0 (socket says 172.21.15.138:59886) 2024-04-27T05:20:51.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f23577fe640 1 -- 172.21.15.138:0/1545091256 learned_addr learned my addr 172.21.15.138:0/1545091256 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:20:51.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f234ffff640 1 -- 172.21.15.138:0/1545091256 >> v1:172.21.15.114:6789/0 conn(0x7f2358114490 legacy=0x7f2358114870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f2356ffd640 1 -- 172.21.15.138:0/1545091256 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869788704 0 0) 0x7f2358119bb0 con 0x7f2358114490 2024-04-27T05:20:51.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.570+0000 7f234ffff640 1 -- 172.21.15.138:0/1545091256 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235811ae90 con 0x7f2358117280 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.571+0000 7f234ffff640 1 --2- 172.21.15.138:0/1545091256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358117280 0x7f2358119670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2340009720 tx=0x7f234002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d45befa26e5d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.571+0000 7f2356ffd640 1 -- 172.21.15.138:0/1545091256 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f234002e070 con 0x7f2358117280 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.571+0000 7f2356ffd640 1 -- 172.21.15.138:0/1545091256 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23400316d0 con 0x7f2358117280 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.571+0000 7f2356ffd640 1 -- 172.21.15.138:0/1545091256 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2340031990 con 0x7f2358117280 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 -- 172.21.15.138:0/1545091256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358117280 msgr2=0x7f2358119670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 --2- 172.21.15.138:0/1545091256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358117280 0x7f2358119670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2340009720 tx=0x7f234002b840 comp rx=0 tx=0).stop 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 -- 172.21.15.138:0/1545091256 shutdown_connections 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 --2- 172.21.15.138:0/1545091256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358117280 0x7f2358119670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 -- 172.21.15.138:0/1545091256 >> 172.21.15.138:0/1545091256 conn(0x7f235806f910 msgr2=0x7f2358071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 -- 172.21.15.138:0/1545091256 shutdown_connections 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.572+0000 7f235e175640 1 -- 172.21.15.138:0/1545091256 wait complete. 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f235e175640 1 Processor -- start 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f235e175640 1 -- start start 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f235e175640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 0x7f23581a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f235e175640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581a2f50 con 0x7f2358114490 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f23577fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 0x7f23581a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f23577fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 0x7f23581a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:40554/0 (socket says 172.21.15.138:40554) 2024-04-27T05:20:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f23577fe640 1 -- 172.21.15.138:0/94043892 learned_addr learned my addr 172.21.15.138:0/94043892 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f23577fe640 1 -- 172.21.15.138:0/94043892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009400 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.573+0000 7f23577fe640 1 --2- 172.21.15.138:0/94043892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 0x7f23581a2a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f234800c420 tx=0x7f234800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23480074d0 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348007630 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2348004950 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23581a31b0 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23581a71b0 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f234801d440 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.574+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f234801c840 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.575+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235804e9b0 con 0x7f2358114490 2024-04-27T05:20:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.577+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f2348018070 con 0x7f2358114490 2024-04-27T05:20:51.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.664+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd new", "uuid": "224d0382-aff8-4501-95a0-060872ad7f05", "id": 3} v 0) v1 -- 0x7f2358117280 con 0x7f2358114490 2024-04-27T05:20:51.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.666+0000 7f2354ff9640 1 -- 172.21.15.138:0/94043892 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "224d0382-aff8-4501-95a0-060872ad7f05", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f234801c480 con 0x7f2358114490 2024-04-27T05:20:51.669 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-04-27T05:20:51.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 msgr2=0x7f23581a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 --2- 172.21.15.138:0/94043892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 0x7f23581a2a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f234800c420 tx=0x7f234800c8e0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 shutdown_connections 2024-04-27T05:20:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 --2- 172.21.15.138:0/94043892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2358114490 0x7f23581a2a10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 >> 172.21.15.138:0/94043892 conn(0x7f235806f910 msgr2=0x7f2358115fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 shutdown_connections 2024-04-27T05:20:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:20:51.667+0000 7f235e175640 1 -- 172.21.15.138:0/94043892 wait complete. 2024-04-27T05:20:51.680 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-27T05:20:51.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-27T05:20:51.682 INFO:tasks.ceph.osd.0:Started 2024-04-27T05:20:51.682 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-27T05:20:51.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-27T05:20:51.684 INFO:tasks.ceph.osd.1:Started 2024-04-27T05:20:51.684 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-27T05:20:51.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-27T05:20:51.686 INFO:tasks.ceph.osd.2:Started 2024-04-27T05:20:51.687 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-27T05:20:51.687 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-27T05:20:51.689 INFO:tasks.ceph.osd.3:Started 2024-04-27T05:20:51.689 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T05:20:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c6ffd640 1 Processor -- start 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c6ffd640 1 -- start start 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c6ffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f26c00930d0 0x7f26c00934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c6ffd640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c0094f90 con 0x7f26c00939f0 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c6ffd640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c006ba60 con 0x7f26c00930d0 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c4d99640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f26c00930d0 0x7f26c00934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26b7fff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f26c00939f0 0x7f26c0129460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:49960/0 (socket says 172.21.15.114:49960) 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26b7fff640 1 -- 172.21.15.114:0/1866326552 learned_addr learned my addr 172.21.15.114:0/1866326552 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c4d99640 1 -- 172.21.15.114:0/1866326552 >> v1:172.21.15.114:6789/0 conn(0x7f26c00939f0 legacy=0x7f26c0129460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.789+0000 7f26c4d99640 1 -- 172.21.15.114:0/1866326552 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c006a090 con 0x7f26c00930d0 2024-04-27T05:20:51.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.790+0000 7f26c4d99640 1 --2- 172.21.15.114:0/1866326552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c00934b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009720 tx=0x7f26bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b410b3209ad0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.790+0000 7f26b77fe640 1 -- 172.21.15.114:0/1866326552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26bc02e070 con 0x7f26c00930d0 2024-04-27T05:20:51.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.790+0000 7f26b77fe640 1 -- 172.21.15.114:0/1866326552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc0316d0 con 0x7f26c00930d0 2024-04-27T05:20:51.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.790+0000 7f26b77fe640 1 -- 172.21.15.114:0/1866326552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26bc031990 con 0x7f26c00930d0 2024-04-27T05:20:51.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 -- 172.21.15.114:0/1866326552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 msgr2=0x7f26c00934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 --2- 172.21.15.114:0/1866326552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c00934b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009720 tx=0x7f26bc02b840 comp rx=0 tx=0).stop 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 -- 172.21.15.114:0/1866326552 shutdown_connections 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 --2- 172.21.15.114:0/1866326552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c00934b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 -- 172.21.15.114:0/1866326552 >> 172.21.15.114:0/1866326552 conn(0x7f26c0099240 msgr2=0x7f26c0099650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 -- 172.21.15.114:0/1866326552 shutdown_connections 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.791+0000 7f26c6ffd640 1 -- 172.21.15.114:0/1866326552 wait complete. 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c6ffd640 1 Processor -- start 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c6ffd640 1 -- start start 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c6ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c01c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c6ffd640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c008faa0 con 0x7f26c00930d0 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c4d99640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c01c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c4d99640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c01c18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59816/0 (socket says 172.21.15.114:59816) 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.792+0000 7f26c4d99640 1 -- 172.21.15.114:0/239387534 learned_addr learned my addr 172.21.15.114:0/239387534 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:20:51.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26c4d99640 1 -- 172.21.15.114:0/239387534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc009400 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26c4d99640 1 --2- 172.21.15.114:0/239387534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c01c18e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26bc034040 tx=0x7f26bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26bc032830 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc032990 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26bc032c70 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c006da10 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c008ebe0 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f26bc04e750 con 0x7f26c00930d0 2024-04-27T05:20:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.793+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f26bc045b10 con 0x7f26c00930d0 2024-04-27T05:20:51.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.794+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c0061a20 con 0x7f26c00930d0 2024-04-27T05:20:51.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.796+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f26bc045df0 con 0x7f26c00930d0 2024-04-27T05:20:51.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.854+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26c008ee30 con 0x7f26c00930d0 2024-04-27T05:20:51.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26b57fa640 1 -- 172.21.15.114:0/239387534 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7f26bc04eb40 con 0x7f26c00930d0 2024-04-27T05:20:51.857 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:20:51.857 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":6,"fsid":"5659e27f-2999-4d49-8a80-9d85e921a02f","created":"2024-04-27T05:20:50.587434+0000","modified":"2024-04-27T05:20:51.666882+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T05:20:51.666882+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9b51ec11-b8cb-42da-a911-1aa825f7a690","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b2e00dd-46de-4ed6-8ea8-cd251b08e55d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b957c5fa-4cab-40a2-bf4d-e6fe710c7d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"224d0382-aff8-4501-95a0-060872ad7f05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 msgr2=0x7f26c01c18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 --2- 172.21.15.114:0/239387534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c01c18e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26bc034040 tx=0x7f26bc0046c0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 shutdown_connections 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 --2- 172.21.15.114:0/239387534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26c00930d0 0x7f26c01c18e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 >> 172.21.15.114:0/239387534 conn(0x7f26c0099240 msgr2=0x7f26c0095900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 shutdown_connections 2024-04-27T05:20:51.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:20:51.855+0000 7f26c6ffd640 1 -- 172.21.15.114:0/239387534 wait complete. 2024-04-27T05:20:51.868 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-27T05:20:51.868 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-27T05:20:52.129 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.127+0000 7f070da0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T05:20:52.216 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.214+0000 7f070da0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T05:20:52.258 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.256+0000 7f070da0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T05:20:52.269 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-04-27T05:20:52.268+0000 7f7a7cc16640 -1 Falling back to public interface 2024-04-27T05:20:52.271 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T05:20:52.278 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-04-27T05:20:52.276+0000 7f83d19dc640 -1 Falling back to public interface 2024-04-27T05:20:52.288 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-04-27T05:20:52.286+0000 7f4ec781e640 -1 Falling back to public interface 2024-04-27T05:20:52.289 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-04-27T05:20:52.286+0000 7f0ee4e14640 -1 Falling back to public interface 2024-04-27T05:20:52.359 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf3384a640 1 Processor -- start 2024-04-27T05:20:52.359 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf3384a640 1 -- start start 2024-04-27T05:20:52.359 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf3384a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdf2c12b780 0x7fdf2c130010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf3384a640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c093d60 con 0x7fdf2c12ac90 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf3384a640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c12bbc0 con 0x7fdf2c12b780 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf30de5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdf2c12b780 0x7fdf2c130010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf315e6640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdf2c12ac90 0x7fdf2c12b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:49976/0 (socket says 172.21.15.114:49976) 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf30de5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdf2c12b780 0x7fdf2c130010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59830/0 (socket says 172.21.15.114:59830) 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.357+0000 7fdf315e6640 1 -- 172.21.15.114:0/787493338 learned_addr learned my addr 172.21.15.114:0/787493338 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:20:52.361 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.358+0000 7fdf30de5640 1 -- 172.21.15.114:0/787493338 >> v1:172.21.15.114:6789/0 conn(0x7fdf2c12ac90 legacy=0x7fdf2c12b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.358+0000 7fdf23fff640 1 -- 172.21.15.114:0/787493338 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255748473 0 0) 0x7fdf2c093d60 con 0x7fdf2c12ac90 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.358+0000 7fdf30de5640 1 -- 172.21.15.114:0/787493338 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0943d0 con 0x7fdf2c12b780 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.358+0000 7fdf30de5640 1 --2- 172.21.15.114:0/787493338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12b780 0x7fdf2c130010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009720 tx=0x7fdf1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ed15339e6234f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.358+0000 7fdf23fff640 1 -- 172.21.15.114:0/787493338 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf1c02e070 con 0x7fdf2c12b780 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf23fff640 1 -- 172.21.15.114:0/787493338 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0316d0 con 0x7fdf2c12b780 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf23fff640 1 -- 172.21.15.114:0/787493338 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf1c031990 con 0x7fdf2c12b780 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 -- 172.21.15.114:0/787493338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12b780 msgr2=0x7fdf2c130010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 --2- 172.21.15.114:0/787493338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12b780 0x7fdf2c130010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009720 tx=0x7fdf1c02b840 comp rx=0 tx=0).stop 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 -- 172.21.15.114:0/787493338 shutdown_connections 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 --2- 172.21.15.114:0/787493338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12b780 0x7fdf2c130010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 -- 172.21.15.114:0/787493338 >> 172.21.15.114:0/787493338 conn(0x7fdf2c0990e0 msgr2=0x7fdf2c0994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 -- 172.21.15.114:0/787493338 shutdown_connections 2024-04-27T05:20:52.362 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.359+0000 7fdf3384a640 1 -- 172.21.15.114:0/787493338 wait complete. 2024-04-27T05:20:52.366 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf3384a640 1 Processor -- start 2024-04-27T05:20:52.366 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf3384a640 1 -- start start 2024-04-27T05:20:52.366 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf3384a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 0x7fdf2c122a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:52.366 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf3384a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c097510 con 0x7fdf2c12ac90 2024-04-27T05:20:52.366 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf315e6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 0x7fdf2c122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf315e6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 0x7fdf2c122a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59842/0 (socket says 172.21.15.114:59842) 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.360+0000 7fdf315e6640 1 -- 172.21.15.114:0/3977784542 learned_addr learned my addr 172.21.15.114:0/3977784542 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf315e6640 1 -- 172.21.15.114:0/3977784542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c009400 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf315e6640 1 --2- 172.21.15.114:0/3977784542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 0x7fdf2c122a10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf28010ae0 tx=0x7fdf2800e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf2800eb90 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2800ecf0 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c123a70 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf2800b0d0 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c125b20 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdf2800b530 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.361+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fdf280208c0 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.362+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c061a20 con 0x7fdf2c12ac90 2024-04-27T05:20:52.367 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.364+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fdf2801c070 con 0x7fdf2c12ac90 2024-04-27T05:20:52.423 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.421+0000 7f070da0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T05:20:52.424 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.422+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2c1240b0 con 0x7fdf2c12ac90 2024-04-27T05:20:52.424 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.422+0000 7fdf21ffb640 1 -- 172.21.15.114:0/3977784542 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7fdf2800ba70 con 0x7fdf2c12ac90 2024-04-27T05:20:52.424 INFO:teuthology.misc.health.smithi114.stdout: 2024-04-27T05:20:52.424 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":6,"fsid":"5659e27f-2999-4d49-8a80-9d85e921a02f","created":"2024-04-27T05:20:50.587434+0000","modified":"2024-04-27T05:20:51.666882+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T05:20:51.666882+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9b51ec11-b8cb-42da-a911-1aa825f7a690","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b2e00dd-46de-4ed6-8ea8-cd251b08e55d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b957c5fa-4cab-40a2-bf4d-e6fe710c7d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"224d0382-aff8-4501-95a0-060872ad7f05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T05:20:52.425 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 msgr2=0x7fdf2c122a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:52.425 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 --2- 172.21.15.114:0/3977784542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 0x7fdf2c122a10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf28010ae0 tx=0x7fdf2800e060 comp rx=0 tx=0).stop 2024-04-27T05:20:52.425 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 shutdown_connections 2024-04-27T05:20:52.425 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 --2- 172.21.15.114:0/3977784542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf2c12ac90 0x7fdf2c122a10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:52.425 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 >> 172.21.15.114:0/3977784542 conn(0x7fdf2c0990e0 msgr2=0x7fdf2c096270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:52.425 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 shutdown_connections 2024-04-27T05:20:52.426 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:52.423+0000 7fdf3384a640 1 -- 172.21.15.114:0/3977784542 wait complete. 2024-04-27T05:20:52.434 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-27T05:20:52.475 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.473+0000 7f070da0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T05:20:52.526 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.524+0000 7f070da0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T05:20:52.639 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:52.636+0000 7f070da0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T05:20:53.016 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.013+0000 7f070da0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-27T05:20:53.269 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.267+0000 7f070da0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T05:20:53.321 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.318+0000 7f070da0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T05:20:53.436 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.434+0000 7f070da0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-27T05:20:53.707 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.705+0000 7f070da0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T05:20:53.754 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.751+0000 7f070da0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T05:20:53.905 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:53.903+0000 7f070da0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T05:20:54.041 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:54.038+0000 7f070da0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T05:20:54.089 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:54.087+0000 7f070da0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T05:20:54.159 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:54.156+0000 7f070da0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T05:20:54.203 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:54.201+0000 7f070da0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T05:20:54.667 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:54.664+0000 7f070da0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T05:20:54.784 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:54.781+0000 7f070da0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T05:20:55.700 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-04-27T05:20:55.698+0000 7f7a7cc16640 -1 osd.0 0 log_to_monitors true 2024-04-27T05:20:55.792 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-27T05:20:55.790+0000 7f06ae468640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-27T05:20:55.817 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-04-27T05:20:55.815+0000 7f83d19dc640 -1 osd.1 0 log_to_monitors true 2024-04-27T05:20:55.843 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-04-27T05:20:55.840+0000 7f4ec781e640 -1 osd.2 0 log_to_monitors true 2024-04-27T05:20:55.848 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-04-27T05:20:55.846+0000 7f0ee4e14640 -1 osd.3 0 log_to_monitors true 2024-04-27T05:20:57.689 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-04-27T05:20:57.687+0000 7f83cd146640 -1 osd.1 0 waiting for initial osdmap 2024-04-27T05:20:57.699 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-04-27T05:20:57.697+0000 7f83c5779640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T05:20:57.713 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-04-27T05:20:57.712+0000 7f7a77e2c640 -1 osd.0 0 waiting for initial osdmap 2024-04-27T05:20:57.724 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-04-27T05:20:57.723+0000 7f7a7045f640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T05:20:58.838 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.926+0000 7f6d48e4b640 1 Processor -- start 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.926+0000 7f6d48e4b640 1 -- start start 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.926+0000 7f6d48e4b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d4412ce70 0x7f6d4412f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.926+0000 7f6d48e4b640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d44126580 con 0x7f6d44126a20 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.926+0000 7f6d48e4b640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d44069e80 con 0x7f6d4412ce70 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.926+0000 7f6d3bfff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d4412ce70 0x7f6d4412f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.927+0000 7f6d3bfff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6d4412ce70 0x7f6d4412f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35268/0 (socket says 172.21.15.114:35268) 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.927+0000 7f6d3bfff640 1 -- 172.21.15.114:0/3514236981 learned_addr learned my addr 172.21.15.114:0/3514236981 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.927+0000 7f6d4259c640 1 --1- 172.21.15.114:0/3514236981 >> v1:172.21.15.114:6789/0 conn(0x7f6d44126a20 0x7f6d44126e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:37950/0 (socket says 172.21.15.114:37950) 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.927+0000 7f6d3bfff640 1 -- 172.21.15.114:0/3514236981 >> v1:172.21.15.114:6789/0 conn(0x7f6d44126a20 legacy=0x7f6d44126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:58.929 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.927+0000 7f6d3bfff640 1 -- 172.21.15.114:0/3514236981 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44126820 con 0x7f6d4412ce70 2024-04-27T05:20:58.932 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.927+0000 7f6d3bfff640 1 --2- 172.21.15.114:0/3514236981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d4412ce70 0x7f6d4412f260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0097f0 tx=0x7f6d2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392f39bdb233a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:58.932 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d41d9b640 1 -- 172.21.15.114:0/3514236981 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c0317a0 con 0x7f6d4412ce70 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d41d9b640 1 -- 172.21.15.114:0/3514236981 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c031900 con 0x7f6d4412ce70 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d41d9b640 1 -- 172.21.15.114:0/3514236981 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c031bc0 con 0x7f6d4412ce70 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 -- 172.21.15.114:0/3514236981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d4412ce70 msgr2=0x7f6d4412f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 --2- 172.21.15.114:0/3514236981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d4412ce70 0x7f6d4412f260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0097f0 tx=0x7f6d2c02b910 comp rx=0 tx=0).stop 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 -- 172.21.15.114:0/3514236981 shutdown_connections 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 --2- 172.21.15.114:0/3514236981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d4412ce70 0x7f6d4412f260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 -- 172.21.15.114:0/3514236981 >> 172.21.15.114:0/3514236981 conn(0x7f6d440990e0 msgr2=0x7f6d440994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 -- 172.21.15.114:0/3514236981 shutdown_connections 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.928+0000 7f6d48e4b640 1 -- 172.21.15.114:0/3514236981 wait complete. 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.929+0000 7f6d48e4b640 1 Processor -- start 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.929+0000 7f6d48e4b640 1 -- start start 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.929+0000 7f6d48e4b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 0x7f6d4412f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.929+0000 7f6d48e4b640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d44124a50 con 0x7f6d44126a20 2024-04-27T05:20:58.933 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d4259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 0x7f6d4412f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d4259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 0x7f6d4412f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35282/0 (socket says 172.21.15.114:35282) 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d4259c640 1 -- 172.21.15.114:0/4268216642 learned_addr learned my addr 172.21.15.114:0/4268216642 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d4259c640 1 -- 172.21.15.114:0/4268216642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c0094d0 con 0x7f6d44126a20 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d4259c640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 0x7f6d4412f060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c450 tx=0x7f6d3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d34007410 con 0x7f6d44126a20 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34007570 con 0x7f6d44126a20 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d441301e0 con 0x7f6d44126a20 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d34004910 con 0x7f6d44126a20 2024-04-27T05:20:58.934 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.930+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d44121fa0 con 0x7f6d44126a20 2024-04-27T05:20:58.935 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.931+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) v1 ==== 44761+0+0 (secure 0 0 0) 0x7f6d34004a70 con 0x7f6d44126a20 2024-04-27T05:20:58.935 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.931+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d441219e0 con 0x7f6d44126a20 2024-04-27T05:20:58.935 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.931+0000 7f6d3affd640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f6d24037670 0x7f6d24039b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:20:58.935 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.931+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2558+0+0 (secure 0 0 0) 0x7f6d34049390 con 0x7f6d44126a20 2024-04-27T05:20:58.935 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.932+0000 7f6d3bfff640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f6d24037670 0x7f6d24039b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:20:58.935 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.932+0000 7f6d3bfff640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f6d24037670 0x7f6d24039b30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c036000 tx=0x7f6d2c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:20:58.938 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:58.936+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f6d340192c0 con 0x7f6d44126a20 2024-04-27T05:20:59.068 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-04-27T05:20:59.066+0000 7f4ec4101640 -1 osd.2 0 waiting for initial osdmap 2024-04-27T05:20:59.078 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-04-27T05:20:59.076+0000 7f4ebaef9640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T05:20:59.108 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.107+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d441301e0 con 0x7f6d44126a20 2024-04-27T05:20:59.109 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.107+0000 7f6d3affd640 1 -- 172.21.15.114:0/4268216642 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6811 (secure 0 0 0) 0x7f6d441301e0 con 0x7f6d44126a20 2024-04-27T05:20:59.109 INFO:teuthology.misc.health.smithi114.stdout: 2024-04-27T05:20:59.109 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":10,"fsid":"5659e27f-2999-4d49-8a80-9d85e921a02f","created":"2024-04-27T05:20:50.587434+0000","modified":"2024-04-27T05:20:58.849194+0000","last_up_change":"2024-04-27T05:20:57.844510+0000","last_in_change":"2024-04-27T05:20:51.666882+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T05:20:58.818060+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9b51ec11-b8cb-42da-a911-1aa825f7a690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":27747},{"type":"v1","addr":"172.21.15.114:6801","nonce":27747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":27747},{"type":"v1","addr":"172.21.15.114:6803","nonce":27747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":27747},{"type":"v1","addr":"172.21.15.114:6807","nonce":27747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":27747},{"type":"v1","addr":"172.21.15.114:6805","nonce":27747}]},"public_addr":"172.21.15.114:6801/27747","cluster_addr":"172.21.15.114:6803/27747","heartbeat_back_addr":"172.21.15.114:6807/27747","heartbeat_front_addr":"172.21.15.114:6805/27747","state":["exists","up"]},{"osd":1,"uuid":"2b2e00dd-46de-4ed6-8ea8-cd251b08e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29864},{"type":"v1","addr":"172.21.15.138:6801","nonce":29864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29864},{"type":"v1","addr":"172.21.15.138:6803","nonce":29864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29864},{"type":"v1","addr":"172.21.15.138:6807","nonce":29864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29864},{"type":"v1","addr":"172.21.15.138:6805","nonce":29864}]},"public_addr":"172.21.15.138:6801/29864","cluster_addr":"172.21.15.138:6803/29864","heartbeat_back_addr":"172.21.15.138:6807/29864","heartbeat_front_addr":"172.21.15.138:6805/29864","state":["exists","up"]},{"osd":2,"uuid":"b957c5fa-4cab-40a2-bf4d-e6fe710c7d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"224d0382-aff8-4501-95a0-060872ad7f05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.710093+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T05:20:59.111 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f6d24037670 msgr2=0x7f6d24039b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:59.111 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f6d24037670 0x7f6d24039b30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c036000 tx=0x7f6d2c008030 comp rx=0 tx=0).stop 2024-04-27T05:20:59.111 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 msgr2=0x7f6d4412f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:20:59.111 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 0x7f6d4412f060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c450 tx=0x7f6d3400c910 comp rx=0 tx=0).stop 2024-04-27T05:20:59.111 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 shutdown_connections 2024-04-27T05:20:59.112 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f6d24037670 0x7f6d24039b30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:59.112 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 --2- 172.21.15.114:0/4268216642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d44126a20 0x7f6d4412f060 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:20:59.112 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 >> 172.21.15.114:0/4268216642 conn(0x7f6d440990e0 msgr2=0x7f6d44096eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:20:59.112 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 shutdown_connections 2024-04-27T05:20:59.112 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:20:59.109+0000 7f6d48e4b640 1 -- 172.21.15.114:0/4268216642 wait complete. 2024-04-27T05:20:59.121 DEBUG:teuthology.misc:2 of 4 OSDs are up 2024-04-27T05:20:59.122 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-04-27T05:20:59.120+0000 7f0ee0050640 -1 osd.3 0 waiting for initial osdmap 2024-04-27T05:20:59.132 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-04-27T05:20:59.130+0000 7f0ed8683640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T05:21:05.525 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T05:21:05.627 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.624+0000 7fc9ed036640 1 Processor -- start 2024-04-27T05:21:05.627 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.624+0000 7fc9ed036640 1 -- start start 2024-04-27T05:21:05.627 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.624+0000 7fc9ed036640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9e8126a20 0x7fc9e8126e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:05.627 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.624+0000 7fc9ed036640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e8126580 con 0x7fc9e8129b00 2024-04-27T05:21:05.627 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.624+0000 7fc9ed036640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e8069e80 con 0x7fc9e8126a20 2024-04-27T05:21:05.628 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.625+0000 7fc9e6d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9e8126a20 0x7fc9e8126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:05.628 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.625+0000 7fc9e659c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc9e8129b00 0x7fc9e8129f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52922/0 (socket says 172.21.15.114:52922) 2024-04-27T05:21:05.628 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.625+0000 7fc9e6d9d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc9e8126a20 0x7fc9e8126e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49488/0 (socket says 172.21.15.114:49488) 2024-04-27T05:21:05.628 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.625+0000 7fc9e659c640 1 -- 172.21.15.114:0/1629809054 learned_addr learned my addr 172.21.15.114:0/1629809054 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:05.628 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.625+0000 7fc9e6d9d640 1 -- 172.21.15.114:0/1629809054 >> v1:172.21.15.114:6789/0 conn(0x7fc9e8129b00 legacy=0x7fc9e8129f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:05.628 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.625+0000 7fc9e6d9d640 1 -- 172.21.15.114:0/1629809054 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8126820 con 0x7fc9e8126a20 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9e6d9d640 1 --2- 172.21.15.114:0/1629809054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e8126e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009720 tx=0x7fc9d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43d1f27e0a0463a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9e5d9b640 1 -- 172.21.15.114:0/1629809054 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9d802e070 con 0x7fc9e8126a20 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9e5d9b640 1 -- 172.21.15.114:0/1629809054 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80316d0 con 0x7fc9e8126a20 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9e5d9b640 1 -- 172.21.15.114:0/1629809054 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9d8031990 con 0x7fc9e8126a20 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 -- 172.21.15.114:0/1629809054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 msgr2=0x7fc9e8126e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 --2- 172.21.15.114:0/1629809054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e8126e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009720 tx=0x7fc9d802b840 comp rx=0 tx=0).stop 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 -- 172.21.15.114:0/1629809054 shutdown_connections 2024-04-27T05:21:05.633 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 --2- 172.21.15.114:0/1629809054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e8126e00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 -- 172.21.15.114:0/1629809054 >> 172.21.15.114:0/1629809054 conn(0x7fc9e80990e0 msgr2=0x7fc9e80994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 -- 172.21.15.114:0/1629809054 shutdown_connections 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.626+0000 7fc9ed036640 1 -- 172.21.15.114:0/1629809054 wait complete. 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9ed036640 1 Processor -- start 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9ed036640 1 -- start start 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9ed036640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e81bc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9ed036640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8121900 con 0x7fc9e8126a20 2024-04-27T05:21:05.634 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9e6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e81bc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9e6d9d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e81bc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49496/0 (socket says 172.21.15.114:49496) 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9e6d9d640 1 -- 172.21.15.114:0/2211849053 learned_addr learned my addr 172.21.15.114:0/2211849053 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9e6d9d640 1 -- 172.21.15.114:0/2211849053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8009400 con 0x7fc9e8126a20 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.628+0000 7fc9e6d9d640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e81bc150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8034040 tx=0x7fc9d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.629+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9d8032830 con 0x7fc9e8126a20 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.629+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d8032990 con 0x7fc9e8126a20 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.629+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e8121f50 con 0x7fc9e8126a20 2024-04-27T05:21:05.635 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.629+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9d8032c50 con 0x7fc9e8126a20 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.629+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8122710 con 0x7fc9e8126a20 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.630+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc9d804d3f0 con 0x7fc9e8126a20 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.630+0000 7fc9cf7fe640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc9d4037700 0x7fc9d4039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.630+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7fc9d806bc60 con 0x7fc9e8126a20 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.630+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e8061a20 con 0x7fc9e8126a20 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.630+0000 7fc9e659c640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc9d4037700 0x7fc9d4039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:05.636 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.631+0000 7fc9e659c640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc9d4037700 0x7fc9d4039bc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc007fd0 tx=0x7fc9dc007ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:05.637 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.635+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc9d8032db0 con 0x7fc9e8126a20 2024-04-27T05:21:05.809 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.807+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9e80717d0 con 0x7fc9e8126a20 2024-04-27T05:21:05.810 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.808+0000 7fc9cf7fe640 1 -- 172.21.15.114:0/2211849053 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7727 (secure 0 0 0) 0x7fc9d80302f0 con 0x7fc9e8126a20 2024-04-27T05:21:05.810 INFO:teuthology.misc.health.smithi114.stdout: 2024-04-27T05:21:05.810 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":13,"fsid":"5659e27f-2999-4d49-8a80-9d85e921a02f","created":"2024-04-27T05:20:50.587434+0000","modified":"2024-04-27T05:21:01.856262+0000","last_up_change":"2024-04-27T05:20:59.853809+0000","last_in_change":"2024-04-27T05:20:51.666882+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T05:20:58.818060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9b51ec11-b8cb-42da-a911-1aa825f7a690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":27747},{"type":"v1","addr":"172.21.15.114:6801","nonce":27747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":27747},{"type":"v1","addr":"172.21.15.114:6803","nonce":27747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":27747},{"type":"v1","addr":"172.21.15.114:6807","nonce":27747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":27747},{"type":"v1","addr":"172.21.15.114:6805","nonce":27747}]},"public_addr":"172.21.15.114:6801/27747","cluster_addr":"172.21.15.114:6803/27747","heartbeat_back_addr":"172.21.15.114:6807/27747","heartbeat_front_addr":"172.21.15.114:6805/27747","state":["exists","up"]},{"osd":1,"uuid":"2b2e00dd-46de-4ed6-8ea8-cd251b08e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29864},{"type":"v1","addr":"172.21.15.138:6801","nonce":29864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29864},{"type":"v1","addr":"172.21.15.138:6803","nonce":29864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29864},{"type":"v1","addr":"172.21.15.138:6807","nonce":29864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29864},{"type":"v1","addr":"172.21.15.138:6805","nonce":29864}]},"public_addr":"172.21.15.138:6801/29864","cluster_addr":"172.21.15.138:6803/29864","heartbeat_back_addr":"172.21.15.138:6807/29864","heartbeat_front_addr":"172.21.15.138:6805/29864","state":["exists","up"]},{"osd":2,"uuid":"b957c5fa-4cab-40a2-bf4d-e6fe710c7d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":29865},{"type":"v1","addr":"172.21.15.138:6811","nonce":29865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":29865},{"type":"v1","addr":"172.21.15.138:6816","nonce":29865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":29865},{"type":"v1","addr":"172.21.15.138:6823","nonce":29865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":29865},{"type":"v1","addr":"172.21.15.138:6821","nonce":29865}]},"public_addr":"172.21.15.138:6811/29865","cluster_addr":"172.21.15.138:6816/29865","heartbeat_back_addr":"172.21.15.138:6823/29865","heartbeat_front_addr":"172.21.15.138:6821/29865","state":["exists","up"]},{"osd":3,"uuid":"224d0382-aff8-4501-95a0-060872ad7f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":29866},{"type":"v1","addr":"172.21.15.138:6810","nonce":29866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":29866},{"type":"v1","addr":"172.21.15.138:6813","nonce":29866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":29866},{"type":"v1","addr":"172.21.15.138:6820","nonce":29866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6815","nonce":29866},{"type":"v1","addr":"172.21.15.138:6817","nonce":29866}]},"public_addr":"172.21.15.138:6810/29866","cluster_addr":"172.21.15.138:6813/29866","heartbeat_back_addr":"172.21.15.138:6820/29866","heartbeat_front_addr":"172.21.15.138:6817/29866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.710093+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:57.688866+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.809534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.816537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc9d4037700 msgr2=0x7fc9d4039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc9d4037700 0x7fc9d4039bc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc007fd0 tx=0x7fc9dc007ac0 comp rx=0 tx=0).stop 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 msgr2=0x7fc9e81bc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e81bc150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8034040 tx=0x7fc9d80046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 shutdown_connections 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc9d4037700 0x7fc9d4039bc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 --2- 172.21.15.114:0/2211849053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc9e8126a20 0x7fc9e81bc150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 >> 172.21.15.114:0/2211849053 conn(0x7fc9e80990e0 msgr2=0x7fc9e8095a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 shutdown_connections 2024-04-27T05:21:05.812 INFO:teuthology.misc.health.smithi114.stderr:2024-04-27T05:21:05.810+0000 7fc9ed036640 1 -- 172.21.15.114:0/2211849053 wait complete. 2024-04-27T05:21:05.822 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-27T05:21:05.823 INFO:tasks.ceph:Creating RBD pool 2024-04-27T05:21:05.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-27T05:21:05.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f554e21c640 1 Processor -- start 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f554e21c640 1 -- start start 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f554e21c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5548097120 0x7f5548129e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f554e21c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55480947b0 con 0x7f554812a3a0 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f554e21c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554806bf40 con 0x7f5548097120 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f55477fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5548097120 0x7f5548129e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f5546ffd640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f554812a3a0 0x7f5548093610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52930/0 (socket says 172.21.15.114:52930) 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f5546ffd640 1 -- 172.21.15.114:0/1722269967 learned_addr learned my addr 172.21.15.114:0/1722269967 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f55477fe640 1 -- 172.21.15.114:0/1722269967 >> v1:172.21.15.114:6789/0 conn(0x7f554812a3a0 legacy=0x7f5548093610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f55477fe640 1 -- 172.21.15.114:0/1722269967 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554806c160 con 0x7f5548097120 2024-04-27T05:21:05.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.916+0000 7f55467fc640 1 -- 172.21.15.114:0/1722269967 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785819586 0 0) 0x7f55480947b0 con 0x7f554812a3a0 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.917+0000 7f55477fe640 1 --2- 172.21.15.114:0/1722269967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f5548129e60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5538009720 tx=0x7f553802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3a4a3088d1f8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.917+0000 7f55467fc640 1 -- 172.21.15.114:0/1722269967 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f553802e070 con 0x7f5548097120 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.917+0000 7f55467fc640 1 -- 172.21.15.114:0/1722269967 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55380316d0 con 0x7f5548097120 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.917+0000 7f55467fc640 1 -- 172.21.15.114:0/1722269967 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5538031990 con 0x7f5548097120 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 -- 172.21.15.114:0/1722269967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 msgr2=0x7f5548129e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 --2- 172.21.15.114:0/1722269967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f5548129e60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5538009720 tx=0x7f553802b840 comp rx=0 tx=0).stop 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 -- 172.21.15.114:0/1722269967 shutdown_connections 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 --2- 172.21.15.114:0/1722269967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f5548129e60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:05.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 -- 172.21.15.114:0/1722269967 >> 172.21.15.114:0/1722269967 conn(0x7f5548098290 msgr2=0x7f5548095a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 -- 172.21.15.114:0/1722269967 shutdown_connections 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.918+0000 7f554e21c640 1 -- 172.21.15.114:0/1722269967 wait complete. 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f554e21c640 1 Processor -- start 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f554e21c640 1 -- start start 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f554e21c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f55481243f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f554e21c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548091f50 con 0x7f5548097120 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f55477fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f55481243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f55477fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f55481243f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49520/0 (socket says 172.21.15.114:49520) 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f55477fe640 1 -- 172.21.15.114:0/614047991 learned_addr learned my addr 172.21.15.114:0/614047991 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.919+0000 7f55477fe640 1 -- 172.21.15.114:0/614047991 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538009400 con 0x7f5548097120 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.920+0000 7f55477fe640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f55481243f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5538034040 tx=0x7f55380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.920+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55380328b0 con 0x7f5548097120 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.920+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538032a10 con 0x7f5548097120 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.920+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5538032cf0 con 0x7f5548097120 2024-04-27T05:21:05.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.920+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5548126350 con 0x7f5548097120 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.920+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554806fa40 con 0x7f5548097120 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.921+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f553804e450 con 0x7f5548097120 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.921+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5548125e30 con 0x7f5548097120 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.921+0000 7f552ffff640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f5534037700 0x7f5534039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.921+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f553806c1a0 con 0x7f5548097120 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.921+0000 7f5546ffd640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f5534037700 0x7f5534039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:05.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.922+0000 7f5546ffd640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f5534037700 0x7f5534039bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f553c000c00 tx=0x7f553c007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:05.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:05.925+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f553804c940 con 0x7f5548097120 2024-04-27T05:21:06.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.094+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f5548061a20 con 0x7f5548097120 2024-04-27T05:21:06.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.869+0000 7f552ffff640 1 -- 172.21.15.114:0/614047991 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f553804c350 con 0x7f5548097120 2024-04-27T05:21:06.871 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'rbd' created 2024-04-27T05:21:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f5534037700 msgr2=0x7f5534039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:06.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f5534037700 0x7f5534039bc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f553c000c00 tx=0x7f553c007910 comp rx=0 tx=0).stop 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 msgr2=0x7f55481243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f55481243f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5538034040 tx=0x7f55380046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 shutdown_connections 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f5534037700 0x7f5534039bc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 --2- 172.21.15.114:0/614047991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5548097120 0x7f55481243f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 >> 172.21.15.114:0/614047991 conn(0x7f5548098290 msgr2=0x7f5548126aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.871+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 shutdown_connections 2024-04-27T05:21:06.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.872+0000 7f554e21c640 1 -- 172.21.15.114:0/614047991 wait complete. 2024-04-27T05:21:06.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.981+0000 7f202c2ab640 1 Processor -- start 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.981+0000 7f202c2ab640 1 -- start start 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f202c2ab640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f20240949e0 0x7f2024094dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f202c2ab640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202409aa10 con 0x7f2024120050 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f202c2ab640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202406b910 con 0x7f20240949e0 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f202a047640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f20240949e0 0x7f2024094dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f2029846640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2024120050 0x7f20241204d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52938/0 (socket says 172.21.15.114:52938) 2024-04-27T05:21:06.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f2029846640 1 -- 172.21.15.114:0/3032385731 learned_addr learned my addr 172.21.15.114:0/3032385731 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:06.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f202a047640 1 -- 172.21.15.114:0/3032385731 >> v1:172.21.15.114:6789/0 conn(0x7f2024120050 legacy=0x7f20241204d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:06.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f202a047640 1 -- 172.21.15.114:0/3032385731 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024069ec0 con 0x7f20240949e0 2024-04-27T05:21:06.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.982+0000 7f2029045640 1 -- 172.21.15.114:0/3032385731 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057232788 0 0) 0x7f202409aa10 con 0x7f2024120050 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f202a047640 1 --2- 172.21.15.114:0/3032385731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f2024094dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2020009720 tx=0x7f202002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8435fbd9d9ec93 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f2029045640 1 -- 172.21.15.114:0/3032385731 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f202002e070 con 0x7f20240949e0 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f2029045640 1 -- 172.21.15.114:0/3032385731 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20200316d0 con 0x7f20240949e0 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f2029045640 1 -- 172.21.15.114:0/3032385731 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2020031990 con 0x7f20240949e0 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f202c2ab640 1 -- 172.21.15.114:0/3032385731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 msgr2=0x7f2024094dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f202c2ab640 1 --2- 172.21.15.114:0/3032385731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f2024094dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2020009720 tx=0x7f202002b840 comp rx=0 tx=0).stop 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f202c2ab640 1 -- 172.21.15.114:0/3032385731 shutdown_connections 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f202c2ab640 1 --2- 172.21.15.114:0/3032385731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f2024094dc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:06.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.983+0000 7f202c2ab640 1 -- 172.21.15.114:0/3032385731 >> 172.21.15.114:0/3032385731 conn(0x7f20240994c0 msgr2=0x7f20240998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:06.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.984+0000 7f202c2ab640 1 -- 172.21.15.114:0/3032385731 shutdown_connections 2024-04-27T05:21:06.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.984+0000 7f202c2ab640 1 -- 172.21.15.114:0/3032385731 wait complete. 2024-04-27T05:21:06.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.984+0000 7f202c2ab640 1 Processor -- start 2024-04-27T05:21:06.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.984+0000 7f202c2ab640 1 -- start start 2024-04-27T05:21:06.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202c2ab640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f20241c1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:06.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202c2ab640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241c10e0 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202a047640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f20241c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202a047640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f20241c1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49544/0 (socket says 172.21.15.114:49544) 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202a047640 1 -- 172.21.15.114:0/600746658 learned_addr learned my addr 172.21.15.114:0/600746658 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202a047640 1 -- 172.21.15.114:0/600746658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009400 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202a047640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f20241c1310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f202002b880 tx=0x7f20200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2020004160 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20200042c0 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2020032640 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202408e900 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.985+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202408e4c0 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.986+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2020032830 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.986+0000 7f201affd640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f20080376b0 0x7f2008039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.986+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3471+0+0 (secure 0 0 0) 0x7f202006c520 con 0x7f20240949e0 2024-04-27T05:21:06.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.986+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024061a20 con 0x7f20240949e0 2024-04-27T05:21:06.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.987+0000 7f2029846640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f20080376b0 0x7f2008039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:06.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.987+0000 7f2029846640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f20080376b0 0x7f2008039b70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2014000c00 tx=0x7f2014007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:06.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:06.991+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f202004e7b0 con 0x7f20240949e0 2024-04-27T05:21:07.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.161+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f202406c420 con 0x7f20240949e0 2024-04-27T05:21:07.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.871+0000 7f201affd640 1 -- 172.21.15.114:0/600746658 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f202003a380 con 0x7f20240949e0 2024-04-27T05:21:07.874 INFO:teuthology.orchestra.run.smithi114.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-27T05:21:07.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f20080376b0 msgr2=0x7f2008039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:07.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f20080376b0 0x7f2008039b70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2014000c00 tx=0x7f2014007970 comp rx=0 tx=0).stop 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 msgr2=0x7f20241c1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f20241c1310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f202002b880 tx=0x7f20200046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 shutdown_connections 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f20080376b0 0x7f2008039b70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 --2- 172.21.15.114:0/600746658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20240949e0 0x7f20241c1310 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 >> 172.21.15.114:0/600746658 conn(0x7f20240994c0 msgr2=0x7f2024095990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 shutdown_connections 2024-04-27T05:21:07.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.874+0000 7f202c2ab640 1 -- 172.21.15.114:0/600746658 wait complete. 2024-04-27T05:21:07.887 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-27T05:21:07.888 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-27T05:21:07.888 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-27T05:21:07.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-27T05:21:07.889 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf93c8f640 1 Processor -- start 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf93c8f640 1 -- start start 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf93c8f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf8c12cf80 0x7fbf8c12f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf93c8f640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8c06c540 con 0x7fbf8c1202f0 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf93c8f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8c1200f0 con 0x7fbf8c12cf80 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf91a2b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf8c1202f0 0x7fbf8c1206d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52944/0 (socket says 172.21.15.114:52944) 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf91a2b640 1 -- 172.21.15.114:0/1367533751 learned_addr learned my addr 172.21.15.114:0/1367533751 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.981+0000 7fbf9122a640 1 --2- 172.21.15.114:0/1367533751 >> v2:172.21.15.114:3300/0 conn(0x7fbf8c12cf80 0x7fbf8c12f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.982+0000 7fbf90a29640 1 -- 172.21.15.114:0/1367533751 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963607314 0 0) 0x7fbf8c06c540 con 0x7fbf8c1202f0 2024-04-27T05:21:07.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.982+0000 7fbf90a29640 1 -- 172.21.15.114:0/1367533751 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf78003520 con 0x7fbf8c1202f0 2024-04-27T05:21:07.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.982+0000 7fbf9122a640 1 -- 172.21.15.114:0/1367533751 >> v1:172.21.15.114:6789/0 conn(0x7fbf8c1202f0 legacy=0x7fbf8c1206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:07.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.982+0000 7fbf9122a640 1 -- 172.21.15.114:0/1367533751 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c092ab0 con 0x7fbf8c12cf80 2024-04-27T05:21:07.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.982+0000 7fbf9122a640 1 --2- 172.21.15.114:0/1367533751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c12cf80 0x7fbf8c12f370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009720 tx=0x7fbf7c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e01d7037da71ae server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:07.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf90a29640 1 -- 172.21.15.114:0/1367533751 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf7c02e070 con 0x7fbf8c12cf80 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf90a29640 1 -- 172.21.15.114:0/1367533751 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0316d0 con 0x7fbf8c12cf80 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf90a29640 1 -- 172.21.15.114:0/1367533751 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf7c031970 con 0x7fbf8c12cf80 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 -- 172.21.15.114:0/1367533751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c12cf80 msgr2=0x7fbf8c12f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 --2- 172.21.15.114:0/1367533751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c12cf80 0x7fbf8c12f370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009720 tx=0x7fbf7c02b840 comp rx=0 tx=0).stop 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 -- 172.21.15.114:0/1367533751 shutdown_connections 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 --2- 172.21.15.114:0/1367533751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c12cf80 0x7fbf8c12f370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 -- 172.21.15.114:0/1367533751 >> 172.21.15.114:0/1367533751 conn(0x7fbf8c099240 msgr2=0x7fbf8c099650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 -- 172.21.15.114:0/1367533751 shutdown_connections 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.983+0000 7fbf93c8f640 1 -- 172.21.15.114:0/1367533751 wait complete. 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf93c8f640 1 Processor -- start 2024-04-27T05:21:07.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf93c8f640 1 -- start start 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf93c8f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 0x7fbf8c12ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf93c8f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c071430 con 0x7fbf8c1202f0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf91a2b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 0x7fbf8c12ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf91a2b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 0x7fbf8c12ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49568/0 (socket says 172.21.15.114:49568) 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.984+0000 7fbf91a2b640 1 -- 172.21.15.114:0/2727564893 learned_addr learned my addr 172.21.15.114:0/2727564893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf91a2b640 1 -- 172.21.15.114:0/2727564893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c009400 con 0x7fbf8c1202f0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf91a2b640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 0x7fbf8c12ad40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c3b0 tx=0x7fbf8800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf88005c10 con 0x7fbf8c1202f0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88005d70 con 0x7fbf8c1202f0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c089010 con 0x7fbf8c1202f0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf88007220 con 0x7fbf8c1202f0 2024-04-27T05:21:07.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.985+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c070ed0 con 0x7fbf8c1202f0 2024-04-27T05:21:07.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.986+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbf88007380 con 0x7fbf8c1202f0 2024-04-27T05:21:07.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.986+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8c12ca90 con 0x7fbf8c1202f0 2024-04-27T05:21:07.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.986+0000 7fbf827fc640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbf700376b0 0x7fbf70039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:07.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.986+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fbf880495f0 con 0x7fbf8c1202f0 2024-04-27T05:21:07.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.986+0000 7fbf9122a640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbf700376b0 0x7fbf70039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:07.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.987+0000 7fbf9122a640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbf700376b0 0x7fbf70039b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c036000 tx=0x7fbf7c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:07.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:07.991+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fbf88019b40 con 0x7fbf8c1202f0 2024-04-27T05:21:08.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.204+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbf8c061a20 con 0x7fbf8c1202f0 2024-04-27T05:21:08.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.205+0000 7fbf827fc640 1 -- 172.21.15.114:0/2727564893 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+86131 (secure 0 0 0) 0x7fbf88017020 con 0x7fbf8c1202f0 2024-04-27T05:21:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbf700376b0 msgr2=0x7fbf70039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbf700376b0 0x7fbf70039b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c036000 tx=0x7fbf7c008030 comp rx=0 tx=0).stop 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 msgr2=0x7fbf8c12ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 0x7fbf8c12ad40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c3b0 tx=0x7fbf8800c870 comp rx=0 tx=0).stop 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 shutdown_connections 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbf700376b0 0x7fbf70039b70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 --2- 172.21.15.114:0/2727564893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8c1202f0 0x7fbf8c12ad40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 >> 172.21.15.114:0/2727564893 conn(0x7fbf8c099240 msgr2=0x7fbf8c129a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.207+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 shutdown_connections 2024-04-27T05:21:08.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.208+0000 7fbf93c8f640 1 -- 172.21.15.114:0/2727564893 wait complete. 2024-04-27T05:21:08.220 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":29568},{"type":"v1","addr":"172.21.15.138:6825","nonce":29568}]},"active_addr":"172.21.15.138:6825/29568","active_change":"2024-04-27T05:20:54.785114+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":812273029}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2386752178}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":183184038}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":3691915499}]}]} 2024-04-27T05:21:08.222 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-27T05:21:08.222 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-27T05:21:08.222 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.314+0000 7f82006ce640 1 Processor -- start 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f82006ce640 1 -- start start 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f82006ce640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f81f812cf80 0x7f81f812f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f82006ce640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f806c540 con 0x7f81f81202f0 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f82006ce640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f81200f0 con 0x7f81f812cf80 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f81fdc69640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f81f812cf80 0x7f81f812f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f81fe46a640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f81f81202f0 0x7f81f81206d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52956/0 (socket says 172.21.15.114:52956) 2024-04-27T05:21:08.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f81fe46a640 1 -- 172.21.15.114:0/4281738737 learned_addr learned my addr 172.21.15.114:0/4281738737 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f81fdc69640 1 -- 172.21.15.114:0/4281738737 >> v1:172.21.15.114:6789/0 conn(0x7f81f81202f0 legacy=0x7f81f81206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.315+0000 7f81fdc69640 1 -- 172.21.15.114:0/4281738737 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8092ab0 con 0x7f81f812cf80 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.316+0000 7f81fdc69640 1 --2- 172.21.15.114:0/4281738737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f812cf80 0x7f81f812f370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009720 tx=0x7f81e802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c643e975901bf853 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.316+0000 7f81fd468640 1 -- 172.21.15.114:0/4281738737 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81e802e070 con 0x7f81f812cf80 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.316+0000 7f81fd468640 1 -- 172.21.15.114:0/4281738737 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e80316d0 con 0x7f81f812cf80 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.316+0000 7f81fd468640 1 -- 172.21.15.114:0/4281738737 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81e8031990 con 0x7f81f812cf80 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 -- 172.21.15.114:0/4281738737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f812cf80 msgr2=0x7f81f812f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 --2- 172.21.15.114:0/4281738737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f812cf80 0x7f81f812f370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009720 tx=0x7f81e802b840 comp rx=0 tx=0).stop 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 -- 172.21.15.114:0/4281738737 shutdown_connections 2024-04-27T05:21:08.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 --2- 172.21.15.114:0/4281738737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f812cf80 0x7f81f812f370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 -- 172.21.15.114:0/4281738737 >> 172.21.15.114:0/4281738737 conn(0x7f81f8099240 msgr2=0x7f81f8099650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 -- 172.21.15.114:0/4281738737 shutdown_connections 2024-04-27T05:21:08.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.317+0000 7f82006ce640 1 -- 172.21.15.114:0/4281738737 wait complete. 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f82006ce640 1 Processor -- start 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f82006ce640 1 -- start start 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f82006ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 0x7f81f80930d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f82006ce640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f806fc60 con 0x7f81f81202f0 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f81fe46a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 0x7f81f80930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f81fe46a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 0x7f81f80930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49590/0 (socket says 172.21.15.114:49590) 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f81fe46a640 1 -- 172.21.15.114:0/1795628091 learned_addr learned my addr 172.21.15.114:0/1795628091 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f81fe46a640 1 -- 172.21.15.114:0/1795628091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009400 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.318+0000 7f81fe46a640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 0x7f81f80930d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c4a0 tx=0x7f81f400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81f4005c10 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4005d70 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f8126940 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81f40071a0 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f80700a0 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f81f4007300 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.319+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f8096ec0 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.320+0000 7f81eeffd640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f81dc037700 0x7f81dc039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.320+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f81f4049460 con 0x7f81f81202f0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.320+0000 7f81fdc69640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f81dc037700 0x7f81dc039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.320+0000 7f81fdc69640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f81dc037700 0x7f81dc039bc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f81e8036000 tx=0x7f81e8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.324+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f81f40196e0 con 0x7f81f81202f0 2024-04-27T05:21:08.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.494+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81f812c330 con 0x7f81f81202f0 2024-04-27T05:21:08.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.494+0000 7f81eeffd640 1 -- 172.21.15.114:0/1795628091 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+9475 (secure 0 0 0) 0x7f81f4017070 con 0x7f81f81202f0 2024-04-27T05:21:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:21:08.497 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"fsid":"5659e27f-2999-4d49-8a80-9d85e921a02f","created":"2024-04-27T05:20:50.587434+0000","modified":"2024-04-27T05:21:07.868372+0000","last_up_change":"2024-04-27T05:20:59.853809+0000","last_in_change":"2024-04-27T05:20:51.666882+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T05:20:58.818060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T05:21:06.112781+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9b51ec11-b8cb-42da-a911-1aa825f7a690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":27747},{"type":"v1","addr":"172.21.15.114:6801","nonce":27747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":27747},{"type":"v1","addr":"172.21.15.114:6803","nonce":27747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":27747},{"type":"v1","addr":"172.21.15.114:6807","nonce":27747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":27747},{"type":"v1","addr":"172.21.15.114:6805","nonce":27747}]},"public_addr":"172.21.15.114:6801/27747","cluster_addr":"172.21.15.114:6803/27747","heartbeat_back_addr":"172.21.15.114:6807/27747","heartbeat_front_addr":"172.21.15.114:6805/27747","state":["exists","up"]},{"osd":1,"uuid":"2b2e00dd-46de-4ed6-8ea8-cd251b08e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29864},{"type":"v1","addr":"172.21.15.138:6801","nonce":29864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29864},{"type":"v1","addr":"172.21.15.138:6803","nonce":29864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29864},{"type":"v1","addr":"172.21.15.138:6807","nonce":29864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29864},{"type":"v1","addr":"172.21.15.138:6805","nonce":29864}]},"public_addr":"172.21.15.138:6801/29864","cluster_addr":"172.21.15.138:6803/29864","heartbeat_back_addr":"172.21.15.138:6807/29864","heartbeat_front_addr":"172.21.15.138:6805/29864","state":["exists","up"]},{"osd":2,"uuid":"b957c5fa-4cab-40a2-bf4d-e6fe710c7d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":29865},{"type":"v1","addr":"172.21.15.138:6811","nonce":29865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":29865},{"type":"v1","addr":"172.21.15.138:6816","nonce":29865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":29865},{"type":"v1","addr":"172.21.15.138:6823","nonce":29865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":29865},{"type":"v1","addr":"172.21.15.138:6821","nonce":29865}]},"public_addr":"172.21.15.138:6811/29865","cluster_addr":"172.21.15.138:6816/29865","heartbeat_back_addr":"172.21.15.138:6823/29865","heartbeat_front_addr":"172.21.15.138:6821/29865","state":["exists","up"]},{"osd":3,"uuid":"224d0382-aff8-4501-95a0-060872ad7f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":29866},{"type":"v1","addr":"172.21.15.138:6810","nonce":29866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":29866},{"type":"v1","addr":"172.21.15.138:6813","nonce":29866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":29866},{"type":"v1","addr":"172.21.15.138:6820","nonce":29866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6815","nonce":29866},{"type":"v1","addr":"172.21.15.138:6817","nonce":29866}]},"public_addr":"172.21.15.138:6810/29866","cluster_addr":"172.21.15.138:6813/29866","heartbeat_back_addr":"172.21.15.138:6820/29866","heartbeat_front_addr":"172.21.15.138:6817/29866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.710093+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:57.688866+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.809534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.816537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f81dc037700 msgr2=0x7f81dc039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f81dc037700 0x7f81dc039bc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f81e8036000 tx=0x7f81e8008030 comp rx=0 tx=0).stop 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 msgr2=0x7f81f80930d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 0x7f81f80930d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c4a0 tx=0x7f81f400c960 comp rx=0 tx=0).stop 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 shutdown_connections 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f81dc037700 0x7f81dc039bc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 --2- 172.21.15.114:0/1795628091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81f81202f0 0x7f81f80930d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 >> 172.21.15.114:0/1795628091 conn(0x7f81f8099240 msgr2=0x7f81f8095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 shutdown_connections 2024-04-27T05:21:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.496+0000 7f82006ce640 1 -- 172.21.15.114:0/1795628091 wait complete. 2024-04-27T05:21:08.509 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-27T05:21:08.509 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T05:21:08.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.602+0000 7f43a878d640 1 Processor -- start 2024-04-27T05:21:08.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.602+0000 7f43a878d640 1 -- start start 2024-04-27T05:21:08.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a878d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f43a012cf80 0x7f43a012f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a878d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a006c540 con 0x7f43a01202f0 2024-04-27T05:21:08.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a878d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a01200f0 con 0x7f43a012cf80 2024-04-27T05:21:08.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a5d28640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f43a012cf80 0x7f43a012f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a6529640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f43a01202f0 0x7f43a01206d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52962/0 (socket says 172.21.15.114:52962) 2024-04-27T05:21:08.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a6529640 1 -- 172.21.15.114:0/2691354417 learned_addr learned my addr 172.21.15.114:0/2691354417 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:08.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a5d28640 1 -- 172.21.15.114:0/2691354417 >> v1:172.21.15.114:6789/0 conn(0x7f43a01202f0 legacy=0x7f43a01206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.603+0000 7f43a5d28640 1 -- 172.21.15.114:0/2691354417 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0092ab0 con 0x7f43a012cf80 2024-04-27T05:21:08.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.604+0000 7f43a5d28640 1 --2- 172.21.15.114:0/2691354417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a012cf80 0x7f43a012f370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4394009720 tx=0x7f439402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adb203564d30d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.604+0000 7f43a5527640 1 -- 172.21.15.114:0/2691354417 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f439402e070 con 0x7f43a012cf80 2024-04-27T05:21:08.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.604+0000 7f43a5527640 1 -- 172.21.15.114:0/2691354417 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43940316d0 con 0x7f43a012cf80 2024-04-27T05:21:08.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.604+0000 7f43a5527640 1 -- 172.21.15.114:0/2691354417 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4394031990 con 0x7f43a012cf80 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 -- 172.21.15.114:0/2691354417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a012cf80 msgr2=0x7f43a012f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 --2- 172.21.15.114:0/2691354417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a012cf80 0x7f43a012f370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4394009720 tx=0x7f439402b840 comp rx=0 tx=0).stop 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 -- 172.21.15.114:0/2691354417 shutdown_connections 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 --2- 172.21.15.114:0/2691354417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a012cf80 0x7f43a012f370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 -- 172.21.15.114:0/2691354417 >> 172.21.15.114:0/2691354417 conn(0x7f43a0099240 msgr2=0x7f43a0099650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 -- 172.21.15.114:0/2691354417 shutdown_connections 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.605+0000 7f43a878d640 1 -- 172.21.15.114:0/2691354417 wait complete. 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a878d640 1 Processor -- start 2024-04-27T05:21:08.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a878d640 1 -- start start 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a878d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 0x7f43a01be520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a878d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a006e520 con 0x7f43a01202f0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a6529640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 0x7f43a01be520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a6529640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 0x7f43a01be520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49616/0 (socket says 172.21.15.114:49616) 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a6529640 1 -- 172.21.15.114:0/1808734411 learned_addr learned my addr 172.21.15.114:0/1808734411 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a6529640 1 -- 172.21.15.114:0/1808734411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4394009400 con 0x7f43a01202f0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.606+0000 7f43a6529640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 0x7f43a01be520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c4a0 tx=0x7f439c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f439c005c10 con 0x7f43a01202f0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c005d70 con 0x7f43a01202f0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a006f020 con 0x7f43a01202f0 2024-04-27T05:21:08.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f439c0071a0 con 0x7f43a01202f0 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a006ed10 con 0x7f43a01202f0 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f439c007300 con 0x7f43a01202f0 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.607+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a006df60 con 0x7f43a01202f0 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.608+0000 7f4392ffd640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f4384037700 0x7f4384039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.608+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f439c0493f0 con 0x7f43a01202f0 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.608+0000 7f43a5d28640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f4384037700 0x7f4384039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.609+0000 7f43a5d28640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f4384037700 0x7f4384039bc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4394036000 tx=0x7f4394008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.612+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f439c0198b0 con 0x7f43a01202f0 2024-04-27T05:21:08.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.784+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43a0096ec0 con 0x7f43a01202f0 2024-04-27T05:21:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.785+0000 7f4392ffd640 1 -- 172.21.15.114:0/1808734411 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+9475 (secure 0 0 0) 0x7f439c0192c0 con 0x7f43a01202f0 2024-04-27T05:21:08.787 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:21:08.787 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"fsid":"5659e27f-2999-4d49-8a80-9d85e921a02f","created":"2024-04-27T05:20:50.587434+0000","modified":"2024-04-27T05:21:07.868372+0000","last_up_change":"2024-04-27T05:20:59.853809+0000","last_in_change":"2024-04-27T05:20:51.666882+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T05:20:58.818060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T05:21:06.112781+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9b51ec11-b8cb-42da-a911-1aa825f7a690","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":27747},{"type":"v1","addr":"172.21.15.114:6801","nonce":27747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":27747},{"type":"v1","addr":"172.21.15.114:6803","nonce":27747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":27747},{"type":"v1","addr":"172.21.15.114:6807","nonce":27747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":27747},{"type":"v1","addr":"172.21.15.114:6805","nonce":27747}]},"public_addr":"172.21.15.114:6801/27747","cluster_addr":"172.21.15.114:6803/27747","heartbeat_back_addr":"172.21.15.114:6807/27747","heartbeat_front_addr":"172.21.15.114:6805/27747","state":["exists","up"]},{"osd":1,"uuid":"2b2e00dd-46de-4ed6-8ea8-cd251b08e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29864},{"type":"v1","addr":"172.21.15.138:6801","nonce":29864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29864},{"type":"v1","addr":"172.21.15.138:6803","nonce":29864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29864},{"type":"v1","addr":"172.21.15.138:6807","nonce":29864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29864},{"type":"v1","addr":"172.21.15.138:6805","nonce":29864}]},"public_addr":"172.21.15.138:6801/29864","cluster_addr":"172.21.15.138:6803/29864","heartbeat_back_addr":"172.21.15.138:6807/29864","heartbeat_front_addr":"172.21.15.138:6805/29864","state":["exists","up"]},{"osd":2,"uuid":"b957c5fa-4cab-40a2-bf4d-e6fe710c7d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6809","nonce":29865},{"type":"v1","addr":"172.21.15.138:6811","nonce":29865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":29865},{"type":"v1","addr":"172.21.15.138:6816","nonce":29865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":29865},{"type":"v1","addr":"172.21.15.138:6823","nonce":29865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":29865},{"type":"v1","addr":"172.21.15.138:6821","nonce":29865}]},"public_addr":"172.21.15.138:6811/29865","cluster_addr":"172.21.15.138:6816/29865","heartbeat_back_addr":"172.21.15.138:6823/29865","heartbeat_front_addr":"172.21.15.138:6821/29865","state":["exists","up"]},{"osd":3,"uuid":"224d0382-aff8-4501-95a0-060872ad7f05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":29866},{"type":"v1","addr":"172.21.15.138:6810","nonce":29866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":29866},{"type":"v1","addr":"172.21.15.138:6813","nonce":29866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":29866},{"type":"v1","addr":"172.21.15.138:6820","nonce":29866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6815","nonce":29866},{"type":"v1","addr":"172.21.15.138:6817","nonce":29866}]},"public_addr":"172.21.15.138:6810/29866","cluster_addr":"172.21.15.138:6813/29866","heartbeat_back_addr":"172.21.15.138:6820/29866","heartbeat_front_addr":"172.21.15.138:6817/29866","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.710093+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:57.688866+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.809534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T05:20:56.816537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f4384037700 msgr2=0x7f4384039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f4384037700 0x7f4384039bc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4394036000 tx=0x7f4394008030 comp rx=0 tx=0).stop 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 msgr2=0x7f43a01be520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 0x7f43a01be520 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c4a0 tx=0x7f439c00c960 comp rx=0 tx=0).stop 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 shutdown_connections 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f4384037700 0x7f4384039bc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 --2- 172.21.15.114:0/1808734411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43a01202f0 0x7f43a01be520 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 >> 172.21.15.114:0/1808734411 conn(0x7f43a0099240 msgr2=0x7f43a0095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 shutdown_connections 2024-04-27T05:21:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.787+0000 7f43a878d640 1 -- 172.21.15.114:0/1808734411 wait complete. 2024-04-27T05:21:08.800 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-27T05:21:08.800 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-27T05:21:08.801 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-27T05:21:08.801 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-27T05:21:08.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7fd728dc8640 1 Processor -- start 2024-04-27T05:21:08.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7f899ce77640 1 Processor -- start 2024-04-27T05:21:08.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7fe13fc3a640 1 Processor -- start 2024-04-27T05:21:08.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7fd728dc8640 1 -- start start 2024-04-27T05:21:08.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7f899ce77640 1 -- start start 2024-04-27T05:21:08.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7fe13fc3a640 1 -- start start 2024-04-27T05:21:08.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.901+0000 7f197333c640 1 Processor -- start 2024-04-27T05:21:08.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13fc3a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe13812cf80 0x7fe13812f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd728dc8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd72406c060 0x7fd7241325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd728dc8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd724067470 con 0x7fd72406b570 2024-04-27T05:21:08.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd728dc8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd724072e80 con 0x7fd72406c060 2024-04-27T05:21:08.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f899ce77640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f899812cf80 0x7f899812f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f899ce77640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899806c540 con 0x7f89981202f0 2024-04-27T05:21:08.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f899ce77640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89981200f0 con 0x7f899812cf80 2024-04-27T05:21:08.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13fc3a640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13806c540 con 0x7fe1381202f0 2024-04-27T05:21:08.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f197333c640 1 -- start start 2024-04-27T05:21:08.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13fc3a640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1381200f0 con 0x7fe13812cf80 2024-04-27T05:21:08.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13d1d5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe13812cf80 0x7fe13812f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13d1d5640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe13812cf80 0x7fe13812f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49630/0 (socket says 172.21.15.114:49630) 2024-04-27T05:21:08.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13d1d5640 1 -- 172.21.15.114:0/1749623815 learned_addr learned my addr 172.21.15.114:0/1749623815 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd721d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd72406c060 0x7fd7241325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f8995d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f899812cf80 0x7f899812f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f8995d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f899812cf80 0x7f899812f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49658/0 (socket says 172.21.15.114:49658) 2024-04-27T05:21:08.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd721d9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd72406c060 0x7fd7241325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49646/0 (socket says 172.21.15.114:49646) 2024-04-27T05:21:08.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd721d9b640 1 -- 172.21.15.114:0/414551260 learned_addr learned my addr 172.21.15.114:0/414551260 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd72259c640 1 --1- 172.21.15.114:0/414551260 >> v1:172.21.15.114:6789/0 conn(0x7fd72406b570 0x7fd72406b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52974/0 (socket says 172.21.15.114:52974) 2024-04-27T05:21:08.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f8995d9b640 1 -- 172.21.15.114:0/1451500373 learned_addr learned my addr 172.21.15.114:0/1451500373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13d1d5640 1 -- 172.21.15.114:0/1749623815 >> v1:172.21.15.114:6789/0 conn(0x7fe1381202f0 legacy=0x7fe1381206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T05:21:08.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f197333c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f196c098330 0x7f196c136910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fe13d1d5640 1 -- 172.21.15.114:0/1749623815 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138092ab0 con 0x7fe13812cf80 2024-04-27T05:21:08.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f197333c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c098780 con 0x7f196c097840 2024-04-27T05:21:08.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f197333c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c08d270 con 0x7f196c098330 2024-04-27T05:21:08.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f19710d8640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f196c097840 0x7f196c097c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52996/0 (socket says 172.21.15.114:52996) 2024-04-27T05:21:08.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f19710d8640 1 -- 172.21.15.114:0/554801661 learned_addr learned my addr 172.21.15.114:0/554801661 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:08.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7f19708d7640 1 --2- 172.21.15.114:0/554801661 >> v2:172.21.15.114:3300/0 conn(0x7f196c098330 0x7f196c136910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd72159a640 1 -- 172.21.15.114:0/414551260 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367727510 0 0) 0x7fd724067470 con 0x7fd72406b570 2024-04-27T05:21:08.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd72159a640 1 -- 172.21.15.114:0/414551260 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd70c003520 con 0x7fd72406b570 2024-04-27T05:21:08.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.902+0000 7fd721d9b640 1 -- 172.21.15.114:0/414551260 >> v1:172.21.15.114:6789/0 conn(0x7fd72406b570 legacy=0x7fd72406b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7f8995d9b640 1 -- 172.21.15.114:0/1451500373 >> v1:172.21.15.114:6789/0 conn(0x7f89981202f0 legacy=0x7f89981206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7fd721d9b640 1 -- 172.21.15.114:0/414551260 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724094d30 con 0x7fd72406c060 2024-04-27T05:21:08.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7f8995d9b640 1 -- 172.21.15.114:0/1451500373 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998092ab0 con 0x7f899812cf80 2024-04-27T05:21:08.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7f19708d7640 1 -- 172.21.15.114:0/554801661 >> v1:172.21.15.114:6789/0 conn(0x7f196c097840 legacy=0x7f196c097c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7f19708d7640 1 -- 172.21.15.114:0/554801661 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c098fa0 con 0x7f196c098330 2024-04-27T05:21:08.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7fe13d1d5640 1 --2- 172.21.15.114:0/1749623815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13812cf80 0x7fe13812f370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097f0 tx=0x7fe12802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ce9a5144b15fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7fd721d9b640 1 --2- 172.21.15.114:0/414551260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406c060 0x7fd7241325e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7180097f0 tx=0x7fd71802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8357f1a7f81724d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7f19708d7640 1 --2- 172.21.15.114:0/554801661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c098330 0x7f196c136910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f195c009720 tx=0x7f195c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1cf620d05370b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7f8995d9b640 1 --2- 172.21.15.114:0/1451500373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899812cf80 0x7f899812f370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f898c00ba90 tx=0x7f898c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303bb7d980582625 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7fe13c9d4640 1 -- 172.21.15.114:0/1749623815 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1280315e0 con 0x7fe13812cf80 2024-04-27T05:21:08.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7fe13c9d4640 1 -- 172.21.15.114:0/1749623815 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe128031740 con 0x7fe13812cf80 2024-04-27T05:21:08.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.903+0000 7fe13c9d4640 1 -- 172.21.15.114:0/1749623815 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe128031a00 con 0x7fe13812cf80 2024-04-27T05:21:08.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fd72159a640 1 -- 172.21.15.114:0/414551260 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd718030790 con 0x7fd72406c060 2024-04-27T05:21:08.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f899559a640 1 -- 172.21.15.114:0/1451500373 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f898c02edc0 con 0x7f899812cf80 2024-04-27T05:21:08.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fd72159a640 1 -- 172.21.15.114:0/414551260 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7180308f0 con 0x7fd72406c060 2024-04-27T05:21:08.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f899559a640 1 -- 172.21.15.114:0/1451500373 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c036440 con 0x7f899812cf80 2024-04-27T05:21:08.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fd72159a640 1 -- 172.21.15.114:0/414551260 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd718030bb0 con 0x7fd72406c060 2024-04-27T05:21:08.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f899559a640 1 -- 172.21.15.114:0/1451500373 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f898c036700 con 0x7f899812cf80 2024-04-27T05:21:08.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fd728dc8640 1 -- 172.21.15.114:0/414551260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406c060 msgr2=0x7fd7241325e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f1963fff640 1 -- 172.21.15.114:0/554801661 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f195c02e070 con 0x7f196c098330 2024-04-27T05:21:08.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f1963fff640 1 -- 172.21.15.114:0/554801661 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c0316d0 con 0x7f196c098330 2024-04-27T05:21:08.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fd728dc8640 1 --2- 172.21.15.114:0/414551260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406c060 0x7fd7241325e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7180097f0 tx=0x7fd71802b9a0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f1963fff640 1 -- 172.21.15.114:0/554801661 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f195c031990 con 0x7f196c098330 2024-04-27T05:21:08.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1749623815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13812cf80 msgr2=0x7fe13812f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1749623815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13812cf80 0x7fe13812f370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097f0 tx=0x7fe12802b880 comp rx=0 tx=0).stop 2024-04-27T05:21:08.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f899ce77640 1 -- 172.21.15.114:0/1451500373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899812cf80 msgr2=0x7f899812f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7f899ce77640 1 --2- 172.21.15.114:0/1451500373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899812cf80 0x7f899812f370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f898c00ba90 tx=0x7f898c02e0e0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1749623815 shutdown_connections 2024-04-27T05:21:08.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1749623815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13812cf80 0x7fe13812f370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fd728dc8640 1 -- 172.21.15.114:0/414551260 shutdown_connections 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fd728dc8640 1 --2- 172.21.15.114:0/414551260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406c060 0x7fd7241325e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fd728dc8640 1 -- 172.21.15.114:0/414551260 >> 172.21.15.114:0/414551260 conn(0x7fd72409c640 msgr2=0x7fd724098b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 -- 172.21.15.114:0/554801661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c098330 msgr2=0x7f196c136910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 --2- 172.21.15.114:0/554801661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c098330 0x7f196c136910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f195c009720 tx=0x7f195c02b840 comp rx=0 tx=0).stop 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 -- 172.21.15.114:0/554801661 shutdown_connections 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 --2- 172.21.15.114:0/554801661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c098330 0x7f196c136910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 -- 172.21.15.114:0/554801661 >> 172.21.15.114:0/554801661 conn(0x7f196c09f960 msgr2=0x7f196c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.904+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1749623815 >> 172.21.15.114:0/1749623815 conn(0x7fe138099240 msgr2=0x7fe138099650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fd728dc8640 1 -- 172.21.15.114:0/414551260 shutdown_connections 2024-04-27T05:21:08.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f899ce77640 1 -- 172.21.15.114:0/1451500373 shutdown_connections 2024-04-27T05:21:08.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f899ce77640 1 --2- 172.21.15.114:0/1451500373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f899812cf80 0x7f899812f370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f899ce77640 1 -- 172.21.15.114:0/1451500373 >> 172.21.15.114:0/1451500373 conn(0x7f8998099240 msgr2=0x7f8998099650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 -- 172.21.15.114:0/554801661 shutdown_connections 2024-04-27T05:21:08.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1749623815 shutdown_connections 2024-04-27T05:21:08.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fd728dc8640 1 -- 172.21.15.114:0/414551260 wait complete. 2024-04-27T05:21:08.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f899ce77640 1 -- 172.21.15.114:0/1451500373 shutdown_connections 2024-04-27T05:21:08.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f899ce77640 1 -- 172.21.15.114:0/1451500373 wait complete. 2024-04-27T05:21:08.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7f197333c640 1 -- 172.21.15.114:0/554801661 wait complete. 2024-04-27T05:21:08.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.905+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1749623815 wait complete. 2024-04-27T05:21:08.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13fc3a640 1 Processor -- start 2024-04-27T05:21:08.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13fc3a640 1 -- start start 2024-04-27T05:21:08.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899ce77640 1 Processor -- start 2024-04-27T05:21:08.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899ce77640 1 -- start start 2024-04-27T05:21:08.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13fc3a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 0x7fe1381308c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13fc3a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13806cb60 con 0x7fe1381202f0 2024-04-27T05:21:08.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899ce77640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 0x7f8998125b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899ce77640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899806db80 con 0x7f89981202f0 2024-04-27T05:21:08.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fd728dc8640 1 Processor -- start 2024-04-27T05:21:08.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f197333c640 1 Processor -- start 2024-04-27T05:21:08.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13d9d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 0x7fe1381308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f197333c640 1 -- start start 2024-04-27T05:21:08.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13d9d6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 0x7fe1381308c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49672/0 (socket says 172.21.15.114:49672) 2024-04-27T05:21:08.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fe13d9d6640 1 -- 172.21.15.114:0/1781957294 learned_addr learned my addr 172.21.15.114:0/1781957294 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7fd728dc8640 1 -- start start 2024-04-27T05:21:08.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 0x7f8998125b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899659c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 0x7f8998125b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49688/0 (socket says 172.21.15.114:49688) 2024-04-27T05:21:08.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.906+0000 7f899659c640 1 -- 172.21.15.114:0/1632965271 learned_addr learned my addr 172.21.15.114:0/1632965271 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe13d9d6640 1 -- 172.21.15.114:0/1781957294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280094d0 con 0x7fe1381202f0 2024-04-27T05:21:08.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe13d9d6640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 0x7fe1381308c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe13400c3b0 tx=0x7fe13400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f899659c640 1 -- 172.21.15.114:0/1632965271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c00b770 con 0x7f89981202f0 2024-04-27T05:21:08.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fd728dc8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 0x7fd7241210d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fd728dc8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241229a0 con 0x7fd72406b570 2024-04-27T05:21:08.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f197333c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 0x7f196c1253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f197333c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c128990 con 0x7f196c097840 2024-04-27T05:21:08.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fd72259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 0x7fd7241210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fd72259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 0x7fd7241210d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49704/0 (socket says 172.21.15.114:49704) 2024-04-27T05:21:08.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f899659c640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 0x7f8998125b70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f898800c380 tx=0x7f898800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fd72259c640 1 -- 172.21.15.114:0/3865862448 learned_addr learned my addr 172.21.15.114:0/3865862448 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f19710d8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 0x7f196c1253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f19710d8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 0x7f196c1253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49712/0 (socket says 172.21.15.114:49712) 2024-04-27T05:21:08.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f19710d8640 1 -- 172.21.15.114:0/2701945417 learned_addr learned my addr 172.21.15.114:0/2701945417 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:08.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1340116c0 con 0x7fe1381202f0 2024-04-27T05:21:08.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe134011820 con 0x7fe1381202f0 2024-04-27T05:21:08.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe134011b00 con 0x7fe1381202f0 2024-04-27T05:21:08.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13812c330 con 0x7fe1381202f0 2024-04-27T05:21:08.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe138132300 con 0x7fe1381202f0 2024-04-27T05:21:08.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8988005200 con 0x7f89981202f0 2024-04-27T05:21:08.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988005380 con 0x7f89981202f0 2024-04-27T05:21:08.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8988007020 con 0x7f89981202f0 2024-04-27T05:21:08.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7fd72259c640 1 -- 172.21.15.114:0/3865862448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180094d0 con 0x7fd72406b570 2024-04-27T05:21:08.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fd72259c640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 0x7fd7241210d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c3b0 tx=0x7fd71400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7f19710d8640 1 -- 172.21.15.114:0/2701945417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c009400 con 0x7f196c097840 2024-04-27T05:21:08.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8998127350 con 0x7f89981202f0 2024-04-27T05:21:08.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe13806ba90 con 0x7fe1381202f0 2024-04-27T05:21:08.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.907+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899806dd90 con 0x7f89981202f0 2024-04-27T05:21:08.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7f19710d8640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 0x7f196c1253d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f196800c4a0 tx=0x7f196800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe134011c60 con 0x7fe1381202f0 2024-04-27T05:21:08.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7140074d0 con 0x7fd72406b570 2024-04-27T05:21:08.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd724092180 con 0x7fd72406b570 2024-04-27T05:21:08.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd724123570 con 0x7fd72406b570 2024-04-27T05:21:08.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714007650 con 0x7fd72406b570 2024-04-27T05:21:08.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd714004950 con 0x7fd72406b570 2024-04-27T05:21:08.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f899806df20 con 0x7f89981202f0 2024-04-27T05:21:08.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1968005c10 con 0x7f196c097840 2024-04-27T05:21:08.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1968005d70 con 0x7f196c097840 2024-04-27T05:21:08.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.908+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19680071a0 con 0x7f196c097840 2024-04-27T05:21:08.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f89880071a0 con 0x7f89981202f0 2024-04-27T05:21:08.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe12e7fc640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe11c037700 0x7fe11c039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fe1340498d0 con 0x7fe1381202f0 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe12e7fc640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x7fe11c03ce20 0x7fe11c03f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe11c03f930 con 0x7fe11c03ce20 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe13e1d7640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x7fe11c03ce20 0x7fe11c03f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe134012420 con 0x7fe1381202f0 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fe13d1d5640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe11c037700 0x7fe11c039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c095f20 con 0x7f196c097840 2024-04-27T05:21:08.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd714004ab0 con 0x7fd72406b570 2024-04-27T05:21:08.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7f8986ffd640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f8978037750 0x7f8978039c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f8988049360 con 0x7f89981202f0 2024-04-27T05:21:08.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c126c40 con 0x7f196c097840 2024-04-27T05:21:08.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.909+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd708000f10 con 0x7fd72406b570 2024-04-27T05:21:08.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f8986ffd640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x7f897803ce70 0x7f897803f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f897803f980 con 0x7f897803ce70 2024-04-27T05:21:08.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f8995d9b640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f8978037750 0x7f8978039c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89880035f0 con 0x7f89981202f0 2024-04-27T05:21:08.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd712ffd640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd704037750 0x7fd704039c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fe13e1d7640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x7fe11c03ce20 0x7fe11c03f260 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fd714049a60 con 0x7fd72406b570 2024-04-27T05:21:08.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f8996d9d640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x7f897803ce70 0x7f897803f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd721d9b640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd704037750 0x7fd704039c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd712ffd640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x7fd70403ce70 0x7fd70403f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd70403f980 con 0x7fd70403ce70 2024-04-27T05:21:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fe13d1d5640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe11c037700 0x7fe11c039bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe128036000 tx=0x7fe128008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd722d9d640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x7fd70403ce70 0x7fd70403f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f8996d9d640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x7f897803ce70 0x7f897803f2b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd714013400 con 0x7fd72406b570 2024-04-27T05:21:08.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7fd721d9b640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd704037750 0x7fd704039c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd718036000 tx=0x7fd71803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.910+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f196c095da0 con 0x7f196c097840 2024-04-27T05:21:08.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe11c03f930 con 0x7fe11c03ce20 2024-04-27T05:21:08.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f8995d9b640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f8978037750 0x7f8978039c10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f898c038000 tx=0x7f898c032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7fd722d9d640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x7fd70403ce70 0x7fd70403f2b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f1968007300 con 0x7f196c097840 2024-04-27T05:21:08.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f1961ffb640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f1950037750 0x7f1950039c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f1968049410 con 0x7f196c097840 2024-04-27T05:21:08.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f897803f980 con 0x7f897803ce70 2024-04-27T05:21:08.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f1961ffb640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x7f195003ce70 0x7f195003f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:08.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f195003f980 con 0x7f195003ce70 2024-04-27T05:21:08.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f196804ba00 con 0x7f196c097840 2024-04-27T05:21:08.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f19708d7640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f1950037750 0x7f1950039c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7f19718d9640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x7f195003ce70 0x7f195003f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:08.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.911+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd70403f980 con 0x7fd70403ce70 2024-04-27T05:21:08.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.912+0000 7f19708d7640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f1950037750 0x7f1950039c10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f195c009720 tx=0x7f195c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.912+0000 7f19718d9640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x7f195003ce70 0x7f195003f2b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:08.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.913+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f195003f980 con 0x7f195003ce70 2024-04-27T05:21:08.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.925+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f899806ba90 con 0x7f897803ce70 2024-04-27T05:21:08.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.925+0000 7f8986ffd640 1 -- 172.21.15.114:0/1632965271 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8990011040 con 0x7f897803ce70 2024-04-27T05:21:08.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x7f897803ce70 msgr2=0x7f897803f2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x7f897803ce70 0x7f897803f2b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f8978037750 msgr2=0x7f8978039c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f8978037750 0x7f8978039c10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f898c038000 tx=0x7f898c032000 comp rx=0 tx=0).stop 2024-04-27T05:21:08.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 msgr2=0x7f8998125b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 0x7f8998125b70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f898800c380 tx=0x7f898800c840 comp rx=0 tx=0).stop 2024-04-27T05:21:08.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f196c1271c0 con 0x7f195003ce70 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 shutdown_connections 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x7f897803ce70 0x7f897803f2b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f8978037750 0x7f8978039c10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 --2- 172.21.15.114:0/1632965271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89981202f0 0x7f8998125b70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 >> 172.21.15.114:0/1632965271 conn(0x7f8998099240 msgr2=0x7f8998121270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 shutdown_connections 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f899ce77640 1 -- 172.21.15.114:0/1632965271 wait complete. 2024-04-27T05:21:08.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.926+0000 7f1961ffb640 1 -- 172.21.15.114:0/2701945417 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f196c1271c0 con 0x7f195003ce70 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x7f195003ce70 msgr2=0x7f195003f2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x7f195003ce70 0x7f195003f2b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f1950037750 msgr2=0x7f1950039c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f1950037750 0x7f1950039c10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f195c009720 tx=0x7f195c008030 comp rx=0 tx=0).stop 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 msgr2=0x7f196c1253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 0x7f196c1253d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f196800c4a0 tx=0x7f196800c960 comp rx=0 tx=0).stop 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 shutdown_connections 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x7f195003ce70 0x7f195003f2b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f1950037750 0x7f1950039c10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 --2- 172.21.15.114:0/2701945417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f196c097840 0x7f196c1253d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 >> 172.21.15.114:0/2701945417 conn(0x7f196c09f960 msgr2=0x7f196c06b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 shutdown_connections 2024-04-27T05:21:08.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.927+0000 7f197333c640 1 -- 172.21.15.114:0/2701945417 wait complete. 2024-04-27T05:21:08.952 INFO:teuthology.orchestra.run.smithi114.stdout:47244640262 2024-04-27T05:21:08.953 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T05:21:08.953 INFO:teuthology.orchestra.run.smithi114.stdout:47244640262 2024-04-27T05:21:08.954 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T05:21:08.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.940+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe138061a20 con 0x7fe11c03ce20 2024-04-27T05:21:08.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe12e7fc640 1 -- 172.21.15.114:0/1781957294 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe138061a20 con 0x7fe11c03ce20 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x7fe11c03ce20 msgr2=0x7fe11c03f260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x7fe11c03ce20 0x7fe11c03f260 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe11c037700 msgr2=0x7fe11c039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe11c037700 0x7fe11c039bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe128036000 tx=0x7fe128008030 comp rx=0 tx=0).stop 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 msgr2=0x7fe1381308c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 0x7fe1381308c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe13400c3b0 tx=0x7fe13400c870 comp rx=0 tx=0).stop 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 shutdown_connections 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x7fe11c03ce20 0x7fe11c03f260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe11c037700 0x7fe11c039bc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 --2- 172.21.15.114:0/1781957294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1381202f0 0x7fe1381308c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 >> 172.21.15.114:0/1781957294 conn(0x7fe138099240 msgr2=0x7fe138123bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 shutdown_connections 2024-04-27T05:21:08.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.941+0000 7fe13fc3a640 1 -- 172.21.15.114:0/1781957294 wait complete. 2024-04-27T05:21:08.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.942+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd708002b50 con 0x7fd70403ce70 2024-04-27T05:21:08.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.942+0000 7fd712ffd640 1 -- 172.21.15.114:0/3865862448 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd708002b50 con 0x7fd70403ce70 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x7fd70403ce70 msgr2=0x7fd70403f2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x7fd70403ce70 0x7fd70403f2b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd704037750 msgr2=0x7fd704039c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd704037750 0x7fd704039c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd718036000 tx=0x7fd71803b040 comp rx=0 tx=0).stop 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 msgr2=0x7fd7241210d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 0x7fd7241210d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c3b0 tx=0x7fd71400c870 comp rx=0 tx=0).stop 2024-04-27T05:21:08.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 shutdown_connections 2024-04-27T05:21:08.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x7fd70403ce70 0x7fd70403f2b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd704037750 0x7fd704039c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 --2- 172.21.15.114:0/3865862448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd72406b570 0x7fd7241210d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:08.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 >> 172.21.15.114:0/3865862448 conn(0x7fd72409c640 msgr2=0x7fd72408fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:08.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 shutdown_connections 2024-04-27T05:21:08.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:08.943+0000 7fd728dc8640 1 -- 172.21.15.114:0/3865862448 wait complete. 2024-04-27T05:21:08.961 INFO:teuthology.orchestra.run.smithi114.stdout:38654705669 2024-04-27T05:21:08.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T05:21:08.962 INFO:teuthology.orchestra.run.smithi114.stdout:38654705670 2024-04-27T05:21:08.962 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75edb37640 1 Processor -- start 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75edb37640 1 -- start start 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75edb37640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f75e8097990 0x7f75e8097d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75edb37640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e809cbc0 con 0x7f75e80982b0 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75edb37640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e809c900 con 0x7f75e8097990 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75e77fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f75e8097990 0x7f75e8097d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75e6ffd640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f75e80982b0 0x7f75e8095430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53004/0 (socket says 172.21.15.114:53004) 2024-04-27T05:21:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.058+0000 7f75e6ffd640 1 -- 172.21.15.114:0/743680484 learned_addr learned my addr 172.21.15.114:0/743680484 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.059+0000 7f75e77fe640 1 -- 172.21.15.114:0/743680484 >> v1:172.21.15.114:6789/0 conn(0x7f75e80982b0 legacy=0x7f75e8095430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.059+0000 7f75e77fe640 1 -- 172.21.15.114:0/743680484 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8072c50 con 0x7f75e8097990 2024-04-27T05:21:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.059+0000 7f75e67fc640 1 -- 172.21.15.114:0/743680484 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586638463 0 0) 0x7f75e809cbc0 con 0x7f75e80982b0 2024-04-27T05:21:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.060+0000 7f75e77fe640 1 --2- 172.21.15.114:0/743680484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e8097d70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009720 tx=0x7f75d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ee91fab7acbb60 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.060+0000 7f75e67fc640 1 -- 172.21.15.114:0/743680484 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d802e070 con 0x7f75e8097990 2024-04-27T05:21:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.060+0000 7f75e67fc640 1 -- 172.21.15.114:0/743680484 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d80316d0 con 0x7f75e8097990 2024-04-27T05:21:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.060+0000 7f75e67fc640 1 -- 172.21.15.114:0/743680484 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d8031990 con 0x7f75e8097990 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 -- 172.21.15.114:0/743680484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 msgr2=0x7f75e8097d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 --2- 172.21.15.114:0/743680484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e8097d70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009720 tx=0x7f75d802b840 comp rx=0 tx=0).stop 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 -- 172.21.15.114:0/743680484 shutdown_connections 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 --2- 172.21.15.114:0/743680484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e8097d70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 -- 172.21.15.114:0/743680484 >> 172.21.15.114:0/743680484 conn(0x7f75e809fab0 msgr2=0x7f75e809bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 -- 172.21.15.114:0/743680484 shutdown_connections 2024-04-27T05:21:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.061+0000 7f75edb37640 1 -- 172.21.15.114:0/743680484 wait complete. 2024-04-27T05:21:09.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.062+0000 7f75edb37640 1 Processor -- start 2024-04-27T05:21:09.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.062+0000 7f75edb37640 1 -- start start 2024-04-27T05:21:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.063+0000 7f75edb37640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e81c1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.063+0000 7f75edb37640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80718b0 con 0x7f75e8097990 2024-04-27T05:21:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.063+0000 7f75e77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e81c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.063+0000 7f75e77fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e81c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49728/0 (socket says 172.21.15.114:49728) 2024-04-27T05:21:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.063+0000 7f75e77fe640 1 -- 172.21.15.114:0/406618126 learned_addr learned my addr 172.21.15.114:0/406618126 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.063+0000 7f75e77fe640 1 -- 172.21.15.114:0/406618126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8009400 con 0x7f75e8097990 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.064+0000 7f75e77fe640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e81c1050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009720 tx=0x7f75d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.064+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d80328b0 con 0x7f75e8097990 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.064+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8032a10 con 0x7f75e8097990 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.064+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e806fef0 con 0x7f75e8097990 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.064+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d8032cb0 con 0x7f75e8097990 2024-04-27T05:21:09.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.064+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e806e4b0 con 0x7f75e8097990 2024-04-27T05:21:09.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.065+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f75d804e450 con 0x7f75e8097990 2024-04-27T05:21:09.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.065+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e8070160 con 0x7f75e8097990 2024-04-27T05:21:09.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.065+0000 7f75cffff640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f75d40376b0 0x7f75d4039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.065+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f75d806c2c0 con 0x7f75e8097990 2024-04-27T05:21:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.066+0000 7f75e6ffd640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f75d40376b0 0x7f75d4039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.066+0000 7f75e6ffd640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f75d40376b0 0x7f75d4039b70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f75dc000c00 tx=0x7f75dc007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.074+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f75d8032e60 con 0x7f75e8097990 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.100+0000 7f207538d640 1 Processor -- start 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.100+0000 7f207538d640 1 -- start start 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f207538d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2070097990 0x7f2070097d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f207538d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207009cbc0 con 0x7f20700982b0 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f207538d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207009c900 con 0x7f2070097990 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206e7fc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f20700982b0 0x7f2070095430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53016/0 (socket says 172.21.15.114:53016) 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206e7fc640 1 -- 172.21.15.114:0/3776914775 learned_addr learned my addr 172.21.15.114:0/3776914775 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206effd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2070097990 0x7f2070097d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206dffb640 1 -- 172.21.15.114:0/3776914775 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728730237 0 0) 0x7f207009cbc0 con 0x7f20700982b0 2024-04-27T05:21:09.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206dffb640 1 -- 172.21.15.114:0/3776914775 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2058003520 con 0x7f20700982b0 2024-04-27T05:21:09.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206effd640 1 -- 172.21.15.114:0/3776914775 >> v1:172.21.15.114:6789/0 conn(0x7f20700982b0 legacy=0x7f2070095430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.101+0000 7f206dffb640 1 -- 172.21.15.114:0/3776914775 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513706157 0 0) 0x7f2058003520 con 0x7f20700982b0 2024-04-27T05:21:09.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.102+0000 7f206effd640 1 -- 172.21.15.114:0/3776914775 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070072c50 con 0x7f2070097990 2024-04-27T05:21:09.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.102+0000 7fbce171d640 1 Processor -- start 2024-04-27T05:21:09.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.102+0000 7fbce171d640 1 -- start start 2024-04-27T05:21:09.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbce171d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbcdc097a10 0x7fbcdc097df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbce171d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcdc098c90 con 0x7fbcdc099300 2024-04-27T05:21:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbce171d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc09d090 con 0x7fbcdc097a10 2024-04-27T05:21:09.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7f206effd640 1 --2- 172.21.15.114:0/3776914775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2070097990 0x7f2070097d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2060009720 tx=0x7f206002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74361c75c8365b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbcdaffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbcdc097a10 0x7fbcdc097df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbcdaffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbcdc097a10 0x7fbcdc097df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49736/0 (socket says 172.21.15.114:49736) 2024-04-27T05:21:09.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbcda7fc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbcdc099300 0x7fbcdc129b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53026/0 (socket says 172.21.15.114:53026) 2024-04-27T05:21:09.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbcdaffd640 1 -- 172.21.15.114:0/2264339067 learned_addr learned my addr 172.21.15.114:0/2264339067 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:09.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbcdaffd640 1 -- 172.21.15.114:0/2264339067 >> v1:172.21.15.114:6789/0 conn(0x7fbcdc099300 legacy=0x7fbcdc129b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7fbcdaffd640 1 -- 172.21.15.114:0/2264339067 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc09a1c0 con 0x7fbcdc097a10 2024-04-27T05:21:09.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7f206dffb640 1 -- 172.21.15.114:0/3776914775 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20640015a0 con 0x7f2070097990 2024-04-27T05:21:09.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7f206dffb640 1 -- 172.21.15.114:0/3776914775 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206002e050 con 0x7f2070097990 2024-04-27T05:21:09.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.103+0000 7f206dffb640 1 -- 172.21.15.114:0/3776914775 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2060030640 con 0x7f2070097990 2024-04-27T05:21:09.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7fbcdaffd640 1 --2- 172.21.15.114:0/2264339067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc097df0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40097f0 tx=0x7fbcc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83b9ae50184775 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 -- 172.21.15.114:0/3776914775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2070097990 msgr2=0x7f2070097d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7fbcd9ffb640 1 -- 172.21.15.114:0/2264339067 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcc40317a0 con 0x7fbcdc097a10 2024-04-27T05:21:09.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 --2- 172.21.15.114:0/3776914775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2070097990 0x7f2070097d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2060009720 tx=0x7f206002b710 comp rx=0 tx=0).stop 2024-04-27T05:21:09.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 -- 172.21.15.114:0/3776914775 shutdown_connections 2024-04-27T05:21:09.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 --2- 172.21.15.114:0/3776914775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2070097990 0x7f2070097d70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7fbcd9ffb640 1 -- 172.21.15.114:0/2264339067 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4031900 con 0x7fbcdc097a10 2024-04-27T05:21:09.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7fbcd9ffb640 1 -- 172.21.15.114:0/2264339067 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcc4031bc0 con 0x7fbcdc097a10 2024-04-27T05:21:09.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 -- 172.21.15.114:0/3776914775 >> 172.21.15.114:0/3776914775 conn(0x7f207009fab0 msgr2=0x7f207009bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 -- 172.21.15.114:0/3776914775 shutdown_connections 2024-04-27T05:21:09.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.104+0000 7f207538d640 1 -- 172.21.15.114:0/3776914775 wait complete. 2024-04-27T05:21:09.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 -- 172.21.15.114:0/2264339067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 msgr2=0x7fbcdc097df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 --2- 172.21.15.114:0/2264339067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc097df0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40097f0 tx=0x7fbcc402b910 comp rx=0 tx=0).stop 2024-04-27T05:21:09.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 -- 172.21.15.114:0/2264339067 shutdown_connections 2024-04-27T05:21:09.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 --2- 172.21.15.114:0/2264339067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc097df0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 -- 172.21.15.114:0/2264339067 >> 172.21.15.114:0/2264339067 conn(0x7fbcdc09b4b0 msgr2=0x7fbcdc0708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 -- 172.21.15.114:0/2264339067 shutdown_connections 2024-04-27T05:21:09.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7fbce171d640 1 -- 172.21.15.114:0/2264339067 wait complete. 2024-04-27T05:21:09.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7f207538d640 1 Processor -- start 2024-04-27T05:21:09.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.105+0000 7f207538d640 1 -- start start 2024-04-27T05:21:09.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.106+0000 7f207538d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 0x7f20701c1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.106+0000 7f207538d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207006f8d0 con 0x7f20700982b0 2024-04-27T05:21:09.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.106+0000 7f206effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 0x7f20701c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.106+0000 7f206effd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 0x7f20701c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49740/0 (socket says 172.21.15.114:49740) 2024-04-27T05:21:09.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.106+0000 7f206effd640 1 -- 172.21.15.114:0/2529347425 learned_addr learned my addr 172.21.15.114:0/2529347425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:09.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7f206effd640 1 -- 172.21.15.114:0/2529347425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060009400 con 0x7f20700982b0 2024-04-27T05:21:09.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7fbce171d640 1 Processor -- start 2024-04-27T05:21:09.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7fbce171d640 1 -- start start 2024-04-27T05:21:09.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7f206effd640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 0x7f20701c1210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f206002b750 tx=0x7f20600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2060030640 con 0x7f20700982b0 2024-04-27T05:21:09.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060003610 con 0x7f20700982b0 2024-04-27T05:21:09.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207006e730 con 0x7f20700982b0 2024-04-27T05:21:09.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20600038b0 con 0x7f20700982b0 2024-04-27T05:21:09.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.107+0000 7fbce171d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2070070040 con 0x7f20700982b0 2024-04-27T05:21:09.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7fbcdaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7fbce171d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc1a14c0 con 0x7fbcdc097a10 2024-04-27T05:21:09.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7fbcdaffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49750/0 (socket says 172.21.15.114:49750) 2024-04-27T05:21:09.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7fbcdaffd640 1 -- 172.21.15.114:0/4273704375 learned_addr learned my addr 172.21.15.114:0/4273704375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:09.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f2060039050 con 0x7f20700982b0 2024-04-27T05:21:09.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.108+0000 7fbcdaffd640 1 -- 172.21.15.114:0/4273704375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc40094d0 con 0x7fbcdc097a10 2024-04-27T05:21:09.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7f20577fe640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f205c037700 0x7f205c039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7fbcdaffd640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc19f4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40090b0 tx=0x7fbcc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f2060069f80 con 0x7f20700982b0 2024-04-27T05:21:09.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7f206e7fc640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f205c037700 0x7f205c039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcc4031ba0 con 0x7fbcdc097a10 2024-04-27T05:21:09.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc40443f0 con 0x7fbcdc097a10 2024-04-27T05:21:09.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcc4044690 con 0x7fbcdc097a10 2024-04-27T05:21:09.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc1a1f60 con 0x7fbcdc097a10 2024-04-27T05:21:09.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc1a0330 con 0x7fbcdc097a10 2024-04-27T05:21:09.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7f206e7fc640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f205c037700 0x7f205c039bc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20640016d0 tx=0x7f20640010a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.109+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2070061a20 con 0x7f20700982b0 2024-04-27T05:21:09.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.110+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fbcc404d450 con 0x7fbcdc097a10 2024-04-27T05:21:09.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.110+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc061a20 con 0x7fbcdc097a10 2024-04-27T05:21:09.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.110+0000 7fbcd37fe640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbcbc037750 0x7fbcbc039c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.110+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fbcc402d040 con 0x7fbcdc097a10 2024-04-27T05:21:09.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.111+0000 7fbcda7fc640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbcbc037750 0x7fbcbc039c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.111+0000 7fbcda7fc640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbcbc037750 0x7fbcbc039c10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0015f0 tx=0x7fbccc007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.113+0000 7f95f647d640 1 Processor -- start 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.113+0000 7f95f647d640 1 -- start start 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.113+0000 7f95f647d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f95f012d970 0x7f95f0132050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95f647d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f0099590 con 0x7f95f012ccd0 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95f647d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f0099830 con 0x7f95f012d970 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95effff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f95f012ccd0 0x7f95f012d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53040/0 (socket says 172.21.15.114:53040) 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95effff640 1 -- 172.21.15.114:0/1892938844 learned_addr learned my addr 172.21.15.114:0/1892938844 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95ef7fe640 1 --2- 172.21.15.114:0/1892938844 >> v2:172.21.15.114:3300/0 conn(0x7f95f012d970 0x7f95f0132050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95ef7fe640 1 -- 172.21.15.114:0/1892938844 >> v1:172.21.15.114:6789/0 conn(0x7f95f012ccd0 legacy=0x7f95f012d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.114+0000 7f95ef7fe640 1 -- 172.21.15.114:0/1892938844 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0097ce0 con 0x7f95f012d970 2024-04-27T05:21:09.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.115+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f2060037070 con 0x7f20700982b0 2024-04-27T05:21:09.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.115+0000 7f95ef7fe640 1 --2- 172.21.15.114:0/1892938844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012d970 0x7f95f0132050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009720 tx=0x7f95e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b03c5d3dd6ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.123+0000 7f95eeffd640 1 -- 172.21.15.114:0/1892938844 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e402e070 con 0x7f95f012d970 2024-04-27T05:21:09.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.123+0000 7f95eeffd640 1 -- 172.21.15.114:0/1892938844 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e40316d0 con 0x7f95f012d970 2024-04-27T05:21:09.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.123+0000 7f95eeffd640 1 -- 172.21.15.114:0/1892938844 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e4031990 con 0x7f95f012d970 2024-04-27T05:21:09.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.123+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fbcc404b940 con 0x7fbcdc097a10 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 -- 172.21.15.114:0/1892938844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012d970 msgr2=0x7f95f0132050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 --2- 172.21.15.114:0/1892938844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012d970 0x7f95f0132050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009720 tx=0x7f95e402b840 comp rx=0 tx=0).stop 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 -- 172.21.15.114:0/1892938844 shutdown_connections 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 --2- 172.21.15.114:0/1892938844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012d970 0x7f95f0132050 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 -- 172.21.15.114:0/1892938844 >> 172.21.15.114:0/1892938844 conn(0x7f95f009f4a0 msgr2=0x7f95f009f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 -- 172.21.15.114:0/1892938844 shutdown_connections 2024-04-27T05:21:09.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.124+0000 7f95f647d640 1 -- 172.21.15.114:0/1892938844 wait complete. 2024-04-27T05:21:09.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.126+0000 7f95f647d640 1 Processor -- start 2024-04-27T05:21:09.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.126+0000 7f95f647d640 1 -- start start 2024-04-27T05:21:09.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95f647d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 0x7f95f01c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95f647d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f01243d0 con 0x7f95f012ccd0 2024-04-27T05:21:09.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 0x7f95f01c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95effff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 0x7f95f01c18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49768/0 (socket says 172.21.15.114:49768) 2024-04-27T05:21:09.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95effff640 1 -- 172.21.15.114:0/950327414 learned_addr learned my addr 172.21.15.114:0/950327414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:09.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95effff640 1 -- 172.21.15.114:0/950327414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4009400 con 0x7f95f012ccd0 2024-04-27T05:21:09.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95effff640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 0x7f95f01c18e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95e000c4a0 tx=0x7f95e000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e0005c10 con 0x7f95f012ccd0 2024-04-27T05:21:09.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e0005d70 con 0x7f95f012ccd0 2024-04-27T05:21:09.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.127+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f0071060 con 0x7f95f012ccd0 2024-04-27T05:21:09.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.128+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e00071a0 con 0x7f95f012ccd0 2024-04-27T05:21:09.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.128+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f01257f0 con 0x7f95f012ccd0 2024-04-27T05:21:09.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.129+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f0061a20 con 0x7f95f012ccd0 2024-04-27T05:21:09.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.129+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f95e0007300 con 0x7f95f012ccd0 2024-04-27T05:21:09.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.129+0000 7f95ecff9640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95dc037700 0x7f95dc039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:09.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.129+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f95e0049480 con 0x7f95f012ccd0 2024-04-27T05:21:09.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.130+0000 7f95ef7fe640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95dc037700 0x7f95dc039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:09.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.130+0000 7f95ef7fe640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95dc037700 0x7f95dc039bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009720 tx=0x7f95e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:09.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.138+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f95e0010b80 con 0x7f95f012ccd0 2024-04-27T05:21:09.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.269+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f75e806f930 con 0x7f75e8097990 2024-04-27T05:21:09.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.269+0000 7f75cffff640 1 -- 172.21.15.114:0/406618126 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75d803a380 con 0x7f75e8097990 2024-04-27T05:21:09.271 INFO:teuthology.orchestra.run.smithi114.stdout:47244640261 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f75d40376b0 msgr2=0x7f75d4039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f75d40376b0 0x7f75d4039b70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f75dc000c00 tx=0x7f75dc007970 comp rx=0 tx=0).stop 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 msgr2=0x7f75e81c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e81c1050 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009720 tx=0x7f75d80046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 shutdown_connections 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f75d40376b0 0x7f75d4039b70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 --2- 172.21.15.114:0/406618126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75e8097990 0x7f75e81c1050 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 >> 172.21.15.114:0/406618126 conn(0x7f75e809fab0 msgr2=0x7f75e809a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 shutdown_connections 2024-04-27T05:21:09.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.272+0000 7f75edb37640 1 -- 172.21.15.114:0/406618126 wait complete. 2024-04-27T05:21:09.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.2 2024-04-27T05:21:09.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.302+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2070095b10 con 0x7f20700982b0 2024-04-27T05:21:09.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.302+0000 7f20577fe640 1 -- 172.21.15.114:0/2529347425 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2060003a10 con 0x7f20700982b0 2024-04-27T05:21:09.304 INFO:teuthology.orchestra.run.smithi114.stdout:47244640261 2024-04-27T05:21:09.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f205c037700 msgr2=0x7f205c039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f205c037700 0x7f205c039bc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f20640016d0 tx=0x7f20640010a0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 msgr2=0x7f20701c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 0x7f20701c1210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f206002b750 tx=0x7f20600046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 shutdown_connections 2024-04-27T05:21:09.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f205c037700 0x7f205c039bc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 --2- 172.21.15.114:0/2529347425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20700982b0 0x7f20701c1210 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 >> 172.21.15.114:0/2529347425 conn(0x7f207009fab0 msgr2=0x7f207009a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 shutdown_connections 2024-04-27T05:21:09.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.304+0000 7f207538d640 1 -- 172.21.15.114:0/2529347425 wait complete. 2024-04-27T05:21:09.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.309+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbcdc1a19c0 con 0x7fbcdc097a10 2024-04-27T05:21:09.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.309+0000 7fbcd37fe640 1 -- 172.21.15.114:0/4273704375 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbcc404b350 con 0x7fbcdc097a10 2024-04-27T05:21:09.311 INFO:teuthology.orchestra.run.smithi114.stdout:38654705668 2024-04-27T05:21:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbcbc037750 msgr2=0x7fbcbc039c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbcbc037750 0x7fbcbc039c10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0015f0 tx=0x7fbccc007a40 comp rx=0 tx=0).stop 2024-04-27T05:21:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 msgr2=0x7fbcdc19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc19f4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40090b0 tx=0x7fbcc40046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 shutdown_connections 2024-04-27T05:21:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fbcbc037750 0x7fbcbc039c10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 --2- 172.21.15.114:0/4273704375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbcdc097a10 0x7fbcdc19f4f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 >> 172.21.15.114:0/4273704375 conn(0x7fbcdc09b4b0 msgr2=0x7fbcdc06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 shutdown_connections 2024-04-27T05:21:09.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.311+0000 7fbce171d640 1 -- 172.21.15.114:0/4273704375 wait complete. 2024-04-27T05:21:09.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.3 2024-04-27T05:21:09.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-04-27T05:21:09.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.325+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f95f0131e20 con 0x7f95f012ccd0 2024-04-27T05:21:09.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.326+0000 7f95ecff9640 1 -- 172.21.15.114:0/950327414 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95e0017030 con 0x7f95f012ccd0 2024-04-27T05:21:09.327 INFO:teuthology.orchestra.run.smithi114.stdout:38654705668 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95dc037700 msgr2=0x7f95dc039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95dc037700 0x7f95dc039bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009720 tx=0x7f95e4008030 comp rx=0 tx=0).stop 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 msgr2=0x7f95f01c18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 0x7f95f01c18e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95e000c4a0 tx=0x7f95e000c960 comp rx=0 tx=0).stop 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 shutdown_connections 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95dc037700 0x7f95dc039bc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 --2- 172.21.15.114:0/950327414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95f012ccd0 0x7f95f01c18e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 >> 172.21.15.114:0/950327414 conn(0x7f95f009f4a0 msgr2=0x7f95f012b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 shutdown_connections 2024-04-27T05:21:09.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:09.328+0000 7f95f647d640 1 -- 172.21.15.114:0/950327414 wait complete. 2024-04-27T05:21:09.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.1 2024-04-27T05:21:10.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T05:21:10.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T05:21:10.324 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T05:21:10.341 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T05:21:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.389+0000 7fe35762d640 1 Processor -- start 2024-04-27T05:21:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.389+0000 7fe35762d640 1 -- start start 2024-04-27T05:21:10.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.390+0000 7fe35762d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe350098480 0x7fe350132320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.390+0000 7fe35762d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35009cbc0 con 0x7fe350097990 2024-04-27T05:21:10.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.390+0000 7fe35762d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35009c900 con 0x7fe350098480 2024-04-27T05:21:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.390+0000 7fe354bc8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe350098480 0x7fe350132320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.390+0000 7fe3553c9640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe350097990 0x7fe350097d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53044/0 (socket says 172.21.15.114:53044) 2024-04-27T05:21:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.390+0000 7fe3553c9640 1 -- 172.21.15.114:0/1198387222 learned_addr learned my addr 172.21.15.114:0/1198387222 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.391+0000 7fe354bc8640 1 -- 172.21.15.114:0/1198387222 >> v1:172.21.15.114:6789/0 conn(0x7fe350097990 legacy=0x7fe350097d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.391+0000 7fe347fff640 1 -- 172.21.15.114:0/1198387222 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487741201 0 0) 0x7fe35009cbc0 con 0x7fe350097990 2024-04-27T05:21:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.391+0000 7fe354bc8640 1 -- 172.21.15.114:0/1198387222 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350072c50 con 0x7fe350098480 2024-04-27T05:21:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.392+0000 7fe354bc8640 1 --2- 172.21.15.114:0/1198387222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350098480 0x7fe350132320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe340009720 tx=0x7fe34002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6bf05427302dca server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.392+0000 7fe347fff640 1 -- 172.21.15.114:0/1198387222 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe34002e070 con 0x7fe350098480 2024-04-27T05:21:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.392+0000 7fe347fff640 1 -- 172.21.15.114:0/1198387222 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3400316d0 con 0x7fe350098480 2024-04-27T05:21:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.392+0000 7fe347fff640 1 -- 172.21.15.114:0/1198387222 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe340031990 con 0x7fe350098480 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 -- 172.21.15.114:0/1198387222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350098480 msgr2=0x7fe350132320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 --2- 172.21.15.114:0/1198387222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350098480 0x7fe350132320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe340009720 tx=0x7fe34002b840 comp rx=0 tx=0).stop 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 -- 172.21.15.114:0/1198387222 shutdown_connections 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 --2- 172.21.15.114:0/1198387222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350098480 0x7fe350132320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 -- 172.21.15.114:0/1198387222 >> 172.21.15.114:0/1198387222 conn(0x7fe35009fab0 msgr2=0x7fe35009bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 -- 172.21.15.114:0/1198387222 shutdown_connections 2024-04-27T05:21:10.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.393+0000 7fe35762d640 1 -- 172.21.15.114:0/1198387222 wait complete. 2024-04-27T05:21:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.394+0000 7fe35762d640 1 Processor -- start 2024-04-27T05:21:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.394+0000 7fe35762d640 1 -- start start 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe35762d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 0x7fe35012f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe35762d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35006c6a0 con 0x7fe350097990 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe3553c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 0x7fe35012f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe3553c9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 0x7fe35012f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49780/0 (socket says 172.21.15.114:49780) 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe3553c9640 1 -- 172.21.15.114:0/2562534079 learned_addr learned my addr 172.21.15.114:0/2562534079 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe3553c9640 1 -- 172.21.15.114:0/2562534079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340009400 con 0x7fe350097990 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.395+0000 7fe3553c9640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 0x7fe35012f390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c420 tx=0x7fe34c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.396+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe34c0074d0 con 0x7fe350097990 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.396+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35006c370 con 0x7fe350097990 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.396+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c007630 con 0x7fe350097990 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.396+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe34c0074d0 con 0x7fe350097990 2024-04-27T05:21:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.396+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe350131500 con 0x7fe350097990 2024-04-27T05:21:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.397+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe34c0047e0 con 0x7fe350097990 2024-04-27T05:21:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.397+0000 7fe345ffb640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe334037750 0x7fe334039c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.397+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fe34c049410 con 0x7fe350097990 2024-04-27T05:21:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.397+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35006c1c0 con 0x7fe350097990 2024-04-27T05:21:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.398+0000 7fe354bc8640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe334037750 0x7fe334039c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.398+0000 7fe354bc8640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe334037750 0x7fe334039c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe340036000 tx=0x7fe340008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.402+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fe34c018030 con 0x7fe350097990 2024-04-27T05:21:10.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fd98c270640 1 Processor -- start 2024-04-27T05:21:10.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fa047c58640 1 Processor -- start 2024-04-27T05:21:10.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fd98c270640 1 -- start start 2024-04-27T05:21:10.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fa047c58640 1 -- start start 2024-04-27T05:21:10.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fa047c58640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa040097980 0x7fa040097da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fa047c58640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04009f230 con 0x7fa040096e90 2024-04-27T05:21:10.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fd98c270640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd984134470 0x7fd984136860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fd98c270640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd984097390 con 0x7fd984091620 2024-04-27T05:21:10.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fd98c270640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd984095420 con 0x7fd984134470 2024-04-27T05:21:10.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.432+0000 7fa047c58640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa040073350 con 0x7fa040097980 2024-04-27T05:21:10.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fa0451f3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa040097980 0x7fa040097da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fa0451f3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa040097980 0x7fa040097da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49788/0 (socket says 172.21.15.114:49788) 2024-04-27T05:21:10.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fa0451f3640 1 -- 172.21.15.114:0/3772177167 learned_addr learned my addr 172.21.15.114:0/3772177167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fd98980b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd984134470 0x7fd984136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fd98980b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd984134470 0x7fd984136860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49800/0 (socket says 172.21.15.114:49800) 2024-04-27T05:21:10.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fd98980b640 1 -- 172.21.15.114:0/245989244 learned_addr learned my addr 172.21.15.114:0/245989244 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fa0451f3640 1 -- 172.21.15.114:0/3772177167 >> v1:172.21.15.114:6789/0 conn(0x7fa040096e90 legacy=0x7fa040097270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T05:21:10.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fa0451f3640 1 -- 172.21.15.114:0/3772177167 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040098830 con 0x7fa040097980 2024-04-27T05:21:10.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fd98980b640 1 -- 172.21.15.114:0/245989244 >> v1:172.21.15.114:6789/0 conn(0x7fd984091620 legacy=0x7fd984091a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T05:21:10.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.433+0000 7fd98980b640 1 -- 172.21.15.114:0/245989244 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984095000 con 0x7fd984134470 2024-04-27T05:21:10.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fa0451f3640 1 --2- 172.21.15.114:0/3772177167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040097980 0x7fa040097da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0300097f0 tx=0x7fa03002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ce101eb5d24605 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fa0449f2640 1 -- 172.21.15.114:0/3772177167 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0300315e0 con 0x7fa040097980 2024-04-27T05:21:10.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fa0449f2640 1 -- 172.21.15.114:0/3772177167 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa030031740 con 0x7fa040097980 2024-04-27T05:21:10.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fd98980b640 1 --2- 172.21.15.114:0/245989244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984134470 0x7fd984136860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9740097f0 tx=0x7fd97402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52515666eabb09cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fa0449f2640 1 -- 172.21.15.114:0/3772177167 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa030031a00 con 0x7fa040097980 2024-04-27T05:21:10.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fd98900a640 1 -- 172.21.15.114:0/245989244 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740315e0 con 0x7fd984134470 2024-04-27T05:21:10.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fd98900a640 1 -- 172.21.15.114:0/245989244 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974031740 con 0x7fd984134470 2024-04-27T05:21:10.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.434+0000 7fd98900a640 1 -- 172.21.15.114:0/245989244 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd974031a00 con 0x7fd984134470 2024-04-27T05:21:10.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 -- 172.21.15.114:0/3772177167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040097980 msgr2=0x7fa040097da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 --2- 172.21.15.114:0/3772177167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040097980 0x7fa040097da0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0300097f0 tx=0x7fa03002b880 comp rx=0 tx=0).stop 2024-04-27T05:21:10.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 -- 172.21.15.114:0/3772177167 shutdown_connections 2024-04-27T05:21:10.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 --2- 172.21.15.114:0/3772177167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040097980 0x7fa040097da0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 -- 172.21.15.114:0/3772177167 >> 172.21.15.114:0/3772177167 conn(0x7fa04009d700 msgr2=0x7fa04009db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 -- 172.21.15.114:0/245989244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984134470 msgr2=0x7fd984136860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 --2- 172.21.15.114:0/245989244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984134470 0x7fd984136860 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9740097f0 tx=0x7fd97402b880 comp rx=0 tx=0).stop 2024-04-27T05:21:10.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 -- 172.21.15.114:0/3772177167 shutdown_connections 2024-04-27T05:21:10.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fa047c58640 1 -- 172.21.15.114:0/3772177167 wait complete. 2024-04-27T05:21:10.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 -- 172.21.15.114:0/245989244 shutdown_connections 2024-04-27T05:21:10.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 --2- 172.21.15.114:0/245989244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984134470 0x7fd984136860 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 -- 172.21.15.114:0/245989244 >> 172.21.15.114:0/245989244 conn(0x7fd984098210 msgr2=0x7fd984095a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 -- 172.21.15.114:0/245989244 shutdown_connections 2024-04-27T05:21:10.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.435+0000 7fd98c270640 1 -- 172.21.15.114:0/245989244 wait complete. 2024-04-27T05:21:10.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa047c58640 1 Processor -- start 2024-04-27T05:21:10.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa047c58640 1 -- start start 2024-04-27T05:21:10.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fd98c270640 1 Processor -- start 2024-04-27T05:21:10.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fd98c270640 1 -- start start 2024-04-27T05:21:10.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa047c58640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 0x7fa04008d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa047c58640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04008e660 con 0x7fa040096e90 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fd98c270640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 0x7fd984089ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fd98c270640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98408cfe0 con 0x7fd984091620 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98a00c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 0x7fd984089ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98a00c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 0x7fd984089ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49822/0 (socket says 172.21.15.114:49822) 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98a00c640 1 -- 172.21.15.114:0/2606030081 learned_addr learned my addr 172.21.15.114:0/2606030081 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa0459f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 0x7fa04008d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa0459f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 0x7fa04008d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49808/0 (socket says 172.21.15.114:49808) 2024-04-27T05:21:10.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.436+0000 7fa0459f4640 1 -- 172.21.15.114:0/1198505272 learned_addr learned my addr 172.21.15.114:0/1198505272 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fa0459f4640 1 -- 172.21.15.114:0/1198505272 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300094d0 con 0x7fa040096e90 2024-04-27T05:21:10.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98a00c640 1 -- 172.21.15.114:0/2606030081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740094d0 con 0x7fd984091620 2024-04-27T05:21:10.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fa0459f4640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 0x7fa04008d3b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b360 tx=0x7fa03c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa03c011760 con 0x7fa040096e90 2024-04-27T05:21:10.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98a00c640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 0x7fd984089ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd98000c390 tx=0x7fd98000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c0118e0 con 0x7fa040096e90 2024-04-27T05:21:10.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0400711f0 con 0x7fa040096e90 2024-04-27T05:21:10.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980005410 con 0x7fd984091620 2024-04-27T05:21:10.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800035f0 con 0x7fd984091620 2024-04-27T05:21:10.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9800038d0 con 0x7fd984091620 2024-04-27T05:21:10.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98408ee70 con 0x7fd984091620 2024-04-27T05:21:10.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.437+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98408d300 con 0x7fd984091620 2024-04-27T05:21:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.438+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa03c011bc0 con 0x7fa040096e90 2024-04-27T05:21:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.438+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04008eef0 con 0x7fa040096e90 2024-04-27T05:21:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.438+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98408ce10 con 0x7fd984091620 2024-04-27T05:21:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.439+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd980003a30 con 0x7fd984091620 2024-04-27T05:21:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.439+0000 7fd97affd640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd968037700 0x7fd968039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.439+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fd9800495a0 con 0x7fd984091620 2024-04-27T05:21:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.439+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040061a20 con 0x7fa040096e90 2024-04-27T05:21:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.439+0000 7fd98980b640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd968037700 0x7fd968039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.440+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fa03c011d20 con 0x7fa040096e90 2024-04-27T05:21:10.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.440+0000 7fa0367fc640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fa024037780 0x7fa024039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.440+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fa03c049fd0 con 0x7fa040096e90 2024-04-27T05:21:10.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.440+0000 7fa0451f3640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fa024037780 0x7fa024039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.440+0000 7fd98980b640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd968037700 0x7fd968039bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd974036000 tx=0x7fd974008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.440+0000 7fa0451f3640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fa024037780 0x7fa024039c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0300097f0 tx=0x7fa030008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.444+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd980019b40 con 0x7fd984091620 2024-04-27T05:21:10.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.449+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fa03c021cd0 con 0x7fa040096e90 2024-04-27T05:21:10.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.488+0000 7f77e8e3a640 1 Processor -- start 2024-04-27T05:21:10.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.488+0000 7f77e8e3a640 1 -- start start 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e8e3a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f77e4096e90 0x7f77e4097270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e8e3a640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e409f230 con 0x7f77e40977b0 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e8e3a640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e4073350 con 0x7f77e4096e90 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e259c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f77e4096e90 0x7f77e4097270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e259c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f77e4096e90 0x7f77e4097270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49836/0 (socket says 172.21.15.114:49836) 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e1d9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f77e40977b0 0x7f77e4097bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53076/0 (socket says 172.21.15.114:53076) 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e259c640 1 -- 172.21.15.114:0/258259126 learned_addr learned my addr 172.21.15.114:0/258259126 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e259c640 1 -- 172.21.15.114:0/258259126 >> v1:172.21.15.114:6789/0 conn(0x7f77e40977b0 legacy=0x7f77e4097bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.489+0000 7f77e259c640 1 -- 172.21.15.114:0/258259126 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4098830 con 0x7f77e4096e90 2024-04-27T05:21:10.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.490+0000 7f77e259c640 1 --2- 172.21.15.114:0/258259126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e4097270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f77d40097f0 tx=0x7f77d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3021c3c9b0f3696a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.490+0000 7f77e159a640 1 -- 172.21.15.114:0/258259126 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d40317a0 con 0x7f77e4096e90 2024-04-27T05:21:10.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.490+0000 7f77e159a640 1 -- 172.21.15.114:0/258259126 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d4031900 con 0x7f77e4096e90 2024-04-27T05:21:10.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.490+0000 7f77e159a640 1 -- 172.21.15.114:0/258259126 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d4031bc0 con 0x7f77e4096e90 2024-04-27T05:21:10.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 -- 172.21.15.114:0/258259126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 msgr2=0x7f77e4097270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 --2- 172.21.15.114:0/258259126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e4097270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f77d40097f0 tx=0x7f77d402b910 comp rx=0 tx=0).stop 2024-04-27T05:21:10.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 -- 172.21.15.114:0/258259126 shutdown_connections 2024-04-27T05:21:10.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 --2- 172.21.15.114:0/258259126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e4097270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 -- 172.21.15.114:0/258259126 >> 172.21.15.114:0/258259126 conn(0x7f77e409d700 msgr2=0x7f77e409db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 -- 172.21.15.114:0/258259126 shutdown_connections 2024-04-27T05:21:10.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.491+0000 7f77e8e3a640 1 -- 172.21.15.114:0/258259126 wait complete. 2024-04-27T05:21:10.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.492+0000 7f77e8e3a640 1 Processor -- start 2024-04-27T05:21:10.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.493+0000 7f77e8e3a640 1 -- start start 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.493+0000 7f77e8e3a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e41c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.493+0000 7f77e8e3a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4095100 con 0x7f77e4096e90 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.493+0000 7f77e259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e41c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.493+0000 7f77e259c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e41c1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49848/0 (socket says 172.21.15.114:49848) 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.493+0000 7f77e259c640 1 -- 172.21.15.114:0/3339303708 learned_addr learned my addr 172.21.15.114:0/3339303708 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77e259c640 1 -- 172.21.15.114:0/3339303708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40094d0 con 0x7f77e4096e90 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77e259c640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e41c1a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77d4034040 tx=0x7f77d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d403ad00 con 0x7f77e4096e90 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d403ae60 con 0x7f77e4096e90 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d4044560 con 0x7f77e4096e90 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e4094360 con 0x7f77e4096e90 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.494+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e40935d0 con 0x7f77e4096e90 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.495+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f77d404d450 con 0x7f77e4096e90 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.495+0000 7f77d2ffd640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f77c4037700 0x7f77c4039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:10.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.495+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f77d402d040 con 0x7f77e4096e90 2024-04-27T05:21:10.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.496+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e4071250 con 0x7f77e4096e90 2024-04-27T05:21:10.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.496+0000 7f77e1d9b640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f77c4037700 0x7f77c4039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:10.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.496+0000 7f77e1d9b640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f77c4037700 0x7f77c4039bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77d80015f0 tx=0x7f77d8007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:10.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.500+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f77d4030700 con 0x7f77e4096e90 2024-04-27T05:21:10.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.585+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe350061a20 con 0x7fe350097990 2024-04-27T05:21:10.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.585+0000 7fe345ffb640 1 -- 172.21.15.114:0/2562534079 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe34c03b090 con 0x7fe350097990 2024-04-27T05:21:10.587 INFO:teuthology.orchestra.run.smithi114.stdout:47244640261 2024-04-27T05:21:10.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe334037750 msgr2=0x7fe334039c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe334037750 0x7fe334039c10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe340036000 tx=0x7fe340008030 comp rx=0 tx=0).stop 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 msgr2=0x7fe35012f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 0x7fe35012f390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c420 tx=0x7fe34c00c8e0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 shutdown_connections 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe334037750 0x7fe334039c10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 --2- 172.21.15.114:0/2562534079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe350097990 0x7fe35012f390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 >> 172.21.15.114:0/2562534079 conn(0x7fe35009fab0 msgr2=0x7fe35009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 shutdown_connections 2024-04-27T05:21:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.588+0000 7fe35762d640 1 -- 172.21.15.114:0/2562534079 wait complete. 2024-04-27T05:21:10.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.2 2024-04-27T05:21:10.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.627+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd984136d20 con 0x7fd984091620 2024-04-27T05:21:10.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.627+0000 7fd97affd640 1 -- 172.21.15.114:0/2606030081 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd980017020 con 0x7fd984091620 2024-04-27T05:21:10.629 INFO:teuthology.orchestra.run.smithi114.stdout:38654705668 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd968037700 msgr2=0x7fd968039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd968037700 0x7fd968039bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd974036000 tx=0x7fd974008030 comp rx=0 tx=0).stop 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 msgr2=0x7fd984089ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 0x7fd984089ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd98000c390 tx=0x7fd98000c850 comp rx=0 tx=0).stop 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 shutdown_connections 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd968037700 0x7fd968039bc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 --2- 172.21.15.114:0/2606030081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd984091620 0x7fd984089ea0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 >> 172.21.15.114:0/2606030081 conn(0x7fd984098210 msgr2=0x7fd984099ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 shutdown_connections 2024-04-27T05:21:10.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.629+0000 7fd98c270640 1 -- 172.21.15.114:0/2606030081 wait complete. 2024-04-27T05:21:10.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-04-27T05:21:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.674+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa0400711f0 con 0x7fa040096e90 2024-04-27T05:21:10.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.675+0000 7fa0367fc640 1 -- 172.21.15.114:0/1198505272 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa0400711f0 con 0x7fa040096e90 2024-04-27T05:21:10.677 INFO:teuthology.orchestra.run.smithi114.stdout:47244640261 2024-04-27T05:21:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fa024037780 msgr2=0x7fa024039c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fa024037780 0x7fa024039c40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0300097f0 tx=0x7fa030008030 comp rx=0 tx=0).stop 2024-04-27T05:21:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 msgr2=0x7fa04008d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 0x7fa04008d3b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b360 tx=0x7fa03c00b820 comp rx=0 tx=0).stop 2024-04-27T05:21:10.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 shutdown_connections 2024-04-27T05:21:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fa024037780 0x7fa024039c40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 --2- 172.21.15.114:0/1198505272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa040096e90 0x7fa04008d3b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 >> 172.21.15.114:0/1198505272 conn(0x7fa04009d700 msgr2=0x7fa040099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 shutdown_connections 2024-04-27T05:21:10.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.677+0000 7fa047c58640 1 -- 172.21.15.114:0/1198505272 wait complete. 2024-04-27T05:21:10.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640261 for osd.3 2024-04-27T05:21:10.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.702+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f77e4061a20 con 0x7f77e4096e90 2024-04-27T05:21:10.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.702+0000 7f77d2ffd640 1 -- 172.21.15.114:0/3339303708 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77d4030700 con 0x7f77e4096e90 2024-04-27T05:21:10.704 INFO:teuthology.orchestra.run.smithi114.stdout:38654705668 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f77c4037700 msgr2=0x7f77c4039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f77c4037700 0x7f77c4039bc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77d80015f0 tx=0x7f77d8007a40 comp rx=0 tx=0).stop 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 msgr2=0x7f77e41c1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e41c1a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77d4034040 tx=0x7f77d40046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 shutdown_connections 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f77c4037700 0x7f77c4039bc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 --2- 172.21.15.114:0/3339303708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f77e4096e90 0x7f77e41c1a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:10.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 >> 172.21.15.114:0/3339303708 conn(0x7f77e409d700 msgr2=0x7f77e409abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:10.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 shutdown_connections 2024-04-27T05:21:10.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:10.704+0000 7f77e8e3a640 1 -- 172.21.15.114:0/3339303708 wait complete. 2024-04-27T05:21:10.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705668 for osd.1 2024-04-27T05:21:11.606 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T05:21:11.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T05:21:11.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T05:21:11.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41b4e62640 1 Processor -- start 2024-04-27T05:21:11.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41b4e62640 1 -- start start 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41b4e62640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41b0098480 0x7f41b0132320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41b4e62640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b009cbc0 con 0x7f41b0097990 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41b4e62640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b009c900 con 0x7f41b0098480 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41add9b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f41b0098480 0x7f41b0132320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41ae59c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f41b0097990 0x7f41b0097d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53092/0 (socket says 172.21.15.114:53092) 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.704+0000 7f41ae59c640 1 -- 172.21.15.114:0/3734756540 learned_addr learned my addr 172.21.15.114:0/3734756540 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.705+0000 7f41add9b640 1 -- 172.21.15.114:0/3734756540 >> v1:172.21.15.114:6789/0 conn(0x7f41b0097990 legacy=0x7f41b0097d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.705+0000 7f41add9b640 1 -- 172.21.15.114:0/3734756540 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b0072c50 con 0x7f41b0098480 2024-04-27T05:21:11.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.705+0000 7f41ad59a640 1 -- 172.21.15.114:0/3734756540 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982018228 0 0) 0x7f41b009cbc0 con 0x7f41b0097990 2024-04-27T05:21:11.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.705+0000 7f41add9b640 1 --2- 172.21.15.114:0/3734756540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0098480 0x7f41b0132320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009720 tx=0x7f41a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ef4ddaae050ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41ad59a640 1 -- 172.21.15.114:0/3734756540 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41a402e070 con 0x7f41b0098480 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41ad59a640 1 -- 172.21.15.114:0/3734756540 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a40316d0 con 0x7f41b0098480 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41ad59a640 1 -- 172.21.15.114:0/3734756540 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41a4031990 con 0x7f41b0098480 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 -- 172.21.15.114:0/3734756540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0098480 msgr2=0x7f41b0132320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 --2- 172.21.15.114:0/3734756540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0098480 0x7f41b0132320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009720 tx=0x7f41a402b840 comp rx=0 tx=0).stop 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 -- 172.21.15.114:0/3734756540 shutdown_connections 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 --2- 172.21.15.114:0/3734756540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0098480 0x7f41b0132320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 -- 172.21.15.114:0/3734756540 >> 172.21.15.114:0/3734756540 conn(0x7f41b009fab0 msgr2=0x7f41b009bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 -- 172.21.15.114:0/3734756540 shutdown_connections 2024-04-27T05:21:11.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.706+0000 7f41b4e62640 1 -- 172.21.15.114:0/3734756540 wait complete. 2024-04-27T05:21:11.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.707+0000 7f41b4e62640 1 Processor -- start 2024-04-27T05:21:11.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.707+0000 7f41b4e62640 1 -- start start 2024-04-27T05:21:11.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41b4e62640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 0x7f41b012f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41b4e62640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b006c6a0 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41ae59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 0x7f41b012f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41ae59c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 0x7f41b012f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49874/0 (socket says 172.21.15.114:49874) 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41ae59c640 1 -- 172.21.15.114:0/1464293775 learned_addr learned my addr 172.21.15.114:0/1464293775 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41ae59c640 1 -- 172.21.15.114:0/1464293775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4009400 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41ae59c640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 0x7f41b012f390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c420 tx=0x7f41a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41a00074d0 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a0007630 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b006c370 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41a0004950 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.708+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b0131500 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.709+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f41a0004b70 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.710+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b006c1c0 con 0x7f41b0097990 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.710+0000 7f419effd640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f41900376b0 0x7f4190039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.710+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f41a0049710 con 0x7f41b0097990 2024-04-27T05:21:11.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.710+0000 7f41add9b640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f41900376b0 0x7f4190039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.711+0000 7f41add9b640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f41900376b0 0x7f4190039b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41a4036000 tx=0x7f41a4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.714+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f41a000e280 con 0x7f41b0097990 2024-04-27T05:21:11.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.748+0000 7f22fb2e9640 1 Processor -- start 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.748+0000 7f22fb2e9640 1 -- start start 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22fb2e9640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22f4098480 0x7f22f4132320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22fb2e9640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f409cbc0 con 0x7f22f4097990 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22fb2e9640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f409c900 con 0x7f22f4098480 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22f8884640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22f4098480 0x7f22f4132320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22f8884640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f22f4098480 0x7f22f4132320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49880/0 (socket says 172.21.15.114:49880) 2024-04-27T05:21:11.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22f8884640 1 -- 172.21.15.114:0/464469361 learned_addr learned my addr 172.21.15.114:0/464469361 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:11.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22f8884640 1 -- 172.21.15.114:0/464469361 >> v1:172.21.15.114:6789/0 conn(0x7f22f4097990 legacy=0x7f22f4097d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.749+0000 7f22f8884640 1 -- 172.21.15.114:0/464469361 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4072c50 con 0x7f22f4098480 2024-04-27T05:21:11.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22f8884640 1 --2- 172.21.15.114:0/464469361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4098480 0x7f22f4132320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22e40097f0 tx=0x7f22e402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f2fda60e0aa5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22ebfff640 1 -- 172.21.15.114:0/464469361 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e40317c0 con 0x7f22f4098480 2024-04-27T05:21:11.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22ebfff640 1 -- 172.21.15.114:0/464469361 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4031920 con 0x7f22f4098480 2024-04-27T05:21:11.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22ebfff640 1 -- 172.21.15.114:0/464469361 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e4031bc0 con 0x7f22f4098480 2024-04-27T05:21:11.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22fb2e9640 1 -- 172.21.15.114:0/464469361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4098480 msgr2=0x7f22f4132320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22fb2e9640 1 --2- 172.21.15.114:0/464469361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4098480 0x7f22f4132320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22e40097f0 tx=0x7f22e402b930 comp rx=0 tx=0).stop 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22fb2e9640 1 -- 172.21.15.114:0/464469361 shutdown_connections 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22fb2e9640 1 --2- 172.21.15.114:0/464469361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4098480 0x7f22f4132320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.751+0000 7f22fb2e9640 1 -- 172.21.15.114:0/464469361 >> 172.21.15.114:0/464469361 conn(0x7f22f409fab0 msgr2=0x7f22f409bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.752+0000 7f22fb2e9640 1 -- 172.21.15.114:0/464469361 shutdown_connections 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.752+0000 7f22fb2e9640 1 -- 172.21.15.114:0/464469361 wait complete. 2024-04-27T05:21:11.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.753+0000 7f22fb2e9640 1 Processor -- start 2024-04-27T05:21:11.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.753+0000 7f22fb2e9640 1 -- start start 2024-04-27T05:21:11.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22fb2e9640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 0x7f22f412f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22fb2e9640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f406b9c0 con 0x7f22f4097990 2024-04-27T05:21:11.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22f9085640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 0x7f22f412f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22f9085640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 0x7f22f412f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49890/0 (socket says 172.21.15.114:49890) 2024-04-27T05:21:11.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22f9085640 1 -- 172.21.15.114:0/3461900367 learned_addr learned my addr 172.21.15.114:0/3461900367 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:11.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22f9085640 1 -- 172.21.15.114:0/3461900367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40094d0 con 0x7f22f4097990 2024-04-27T05:21:11.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22f9085640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 0x7f22f412f800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c450 tx=0x7f22f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22f0005200 con 0x7f22f4097990 2024-04-27T05:21:11.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0005380 con 0x7f22f4097990 2024-04-27T05:21:11.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.754+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f4131e60 con 0x7f22f4097990 2024-04-27T05:21:11.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.755+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22f00035f0 con 0x7f22f4097990 2024-04-27T05:21:11.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.755+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f406bea0 con 0x7f22f4097990 2024-04-27T05:21:11.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.756+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f22f0003750 con 0x7f22f4097990 2024-04-27T05:21:11.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.756+0000 7f22e9ffb640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f22d8037700 0x7f22d8039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.756+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f22f0049430 con 0x7f22f4097990 2024-04-27T05:21:11.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.756+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f406bb00 con 0x7f22f4097990 2024-04-27T05:21:11.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.756+0000 7f22f8884640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f22d8037700 0x7f22d8039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.757+0000 7f22f8884640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f22d8037700 0x7f22d8039bc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22e4036000 tx=0x7f22e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.764+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f22f0011590 con 0x7f22f4097990 2024-04-27T05:21:11.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.798+0000 7f95e9444640 1 Processor -- start 2024-04-27T05:21:11.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.798+0000 7f95e9444640 1 -- start start 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.798+0000 7f95e9444640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f95e4097980 0x7f95e4097da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.798+0000 7f95e9444640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e409f230 con 0x7f95e4096e90 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.798+0000 7f95e9444640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e4073350 con 0x7f95e4097980 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.798+0000 7f95e27fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f95e4097980 0x7f95e4097da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.799+0000 7f95e27fc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f95e4097980 0x7f95e4097da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49904/0 (socket says 172.21.15.114:49904) 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.799+0000 7f95e27fc640 1 -- 172.21.15.114:0/4215407047 learned_addr learned my addr 172.21.15.114:0/4215407047 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.799+0000 7f95e27fc640 1 -- 172.21.15.114:0/4215407047 >> v1:172.21.15.114:6789/0 conn(0x7f95e4096e90 legacy=0x7f95e4097270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.799+0000 7f95e1ffb640 1 -- 172.21.15.114:0/4215407047 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386252386 0 0) 0x7f95e409f230 con 0x7f95e4096e90 2024-04-27T05:21:11.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.799+0000 7f95e27fc640 1 -- 172.21.15.114:0/4215407047 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4098830 con 0x7f95e4097980 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.800+0000 7f95e27fc640 1 --2- 172.21.15.114:0/4215407047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4097980 0x7f95e4097da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097f0 tx=0x7f95d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88d5389ea9d3282 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.800+0000 7f95e1ffb640 1 -- 172.21.15.114:0/4215407047 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d80317a0 con 0x7f95e4097980 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.800+0000 7f95e1ffb640 1 -- 172.21.15.114:0/4215407047 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8031900 con 0x7f95e4097980 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.800+0000 7f95e1ffb640 1 -- 172.21.15.114:0/4215407047 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d8031bc0 con 0x7f95e4097980 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 -- 172.21.15.114:0/4215407047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4097980 msgr2=0x7f95e4097da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 --2- 172.21.15.114:0/4215407047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4097980 0x7f95e4097da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097f0 tx=0x7f95d802b910 comp rx=0 tx=0).stop 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 -- 172.21.15.114:0/4215407047 shutdown_connections 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 --2- 172.21.15.114:0/4215407047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4097980 0x7f95e4097da0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 -- 172.21.15.114:0/4215407047 >> 172.21.15.114:0/4215407047 conn(0x7f95e409d700 msgr2=0x7f95e409db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 -- 172.21.15.114:0/4215407047 shutdown_connections 2024-04-27T05:21:11.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.801+0000 7f95e9444640 1 -- 172.21.15.114:0/4215407047 wait complete. 2024-04-27T05:21:11.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.802+0000 7f95e9444640 1 Processor -- start 2024-04-27T05:21:11.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.802+0000 7f95e9444640 1 -- start start 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e9444640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 0x7f95e4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e9444640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4124770 con 0x7f95e4096e90 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 0x7f95e4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 0x7f95e4070340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49918/0 (socket says 172.21.15.114:49918) 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e2ffd640 1 -- 172.21.15.114:0/1283269107 learned_addr learned my addr 172.21.15.114:0/1283269107 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e2ffd640 1 -- 172.21.15.114:0/1283269107 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80094d0 con 0x7f95e4096e90 2024-04-27T05:21:11.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.803+0000 7f95e2ffd640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 0x7f95e4070340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95d400c340 tx=0x7f95d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.804+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d4005410 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.804+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d40035f0 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.804+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e412f9c0 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.804+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95d40038d0 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.804+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e4125a40 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.805+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f95d4003a30 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.805+0000 7f95cb7fe640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95d0037700 0x7f95d0039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.805+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f95d4049550 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.805+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4061a20 con 0x7f95e4096e90 2024-04-27T05:21:11.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.805+0000 7f95e27fc640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95d0037700 0x7f95d0039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.806+0000 7f95e27fc640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95d0037700 0x7f95d0039bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097f0 tx=0x7f95d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.810+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f95d4017020 con 0x7f95e4096e90 2024-04-27T05:21:11.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc39adc8640 1 Processor -- start 2024-04-27T05:21:11.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc39adc8640 1 -- start start 2024-04-27T05:21:11.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc39adc8640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc394096e90 0x7fc394097270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc39adc8640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39409f230 con 0x7fc3940977b0 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc39adc8640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc394073350 con 0x7fc394096e90 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc393fff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc3940977b0 0x7fc394097bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53128/0 (socket says 172.21.15.114:53128) 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc393fff640 1 -- 172.21.15.114:0/3052688813 learned_addr learned my addr 172.21.15.114:0/3052688813 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.818+0000 7fc398b64640 1 --2- 172.21.15.114:0/3052688813 >> v2:172.21.15.114:3300/0 conn(0x7fc394096e90 0x7fc394097270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.819+0000 7fc398b64640 1 -- 172.21.15.114:0/3052688813 >> v1:172.21.15.114:6789/0 conn(0x7fc3940977b0 legacy=0x7fc394097bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.819+0000 7fc3937fe640 1 -- 172.21.15.114:0/3052688813 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013166654 0 0) 0x7fc39409f230 con 0x7fc3940977b0 2024-04-27T05:21:11.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.819+0000 7fc398b64640 1 -- 172.21.15.114:0/3052688813 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394098830 con 0x7fc394096e90 2024-04-27T05:21:11.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.819+0000 7fc398b64640 1 --2- 172.21.15.114:0/3052688813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc394097270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc388009720 tx=0x7fc38802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f49016966d3c6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc3937fe640 1 -- 172.21.15.114:0/3052688813 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc38802e070 con 0x7fc394096e90 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc3937fe640 1 -- 172.21.15.114:0/3052688813 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3880316d0 con 0x7fc394096e90 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc3937fe640 1 -- 172.21.15.114:0/3052688813 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc388031990 con 0x7fc394096e90 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 -- 172.21.15.114:0/3052688813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 msgr2=0x7fc394097270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 --2- 172.21.15.114:0/3052688813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc394097270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc388009720 tx=0x7fc38802b840 comp rx=0 tx=0).stop 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 -- 172.21.15.114:0/3052688813 shutdown_connections 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 --2- 172.21.15.114:0/3052688813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc394097270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 -- 172.21.15.114:0/3052688813 >> 172.21.15.114:0/3052688813 conn(0x7fc39409d700 msgr2=0x7fc39409db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 -- 172.21.15.114:0/3052688813 shutdown_connections 2024-04-27T05:21:11.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.820+0000 7fc39adc8640 1 -- 172.21.15.114:0/3052688813 wait complete. 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.821+0000 7fc39adc8640 1 Processor -- start 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.821+0000 7fc39adc8640 1 -- start start 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc39adc8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc3941c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc39adc8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394126890 con 0x7fc394096e90 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc398b64640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc3941c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc398b64640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc3941c17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49940/0 (socket says 172.21.15.114:49940) 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc398b64640 1 -- 172.21.15.114:0/1758362369 learned_addr learned my addr 172.21.15.114:0/1758362369 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc398b64640 1 -- 172.21.15.114:0/1758362369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009400 con 0x7fc394096e90 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.822+0000 7fc398b64640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc3941c17c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3880316a0 tx=0x7fc3880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.823+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc388004160 con 0x7fc394096e90 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.823+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394124f40 con 0x7fc394096e90 2024-04-27T05:21:11.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.823+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3880042c0 con 0x7fc394096e90 2024-04-27T05:21:11.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.823+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc388003610 con 0x7fc394096e90 2024-04-27T05:21:11.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.823+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394125eb0 con 0x7fc394096e90 2024-04-27T05:21:11.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.824+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fc388043070 con 0x7fc394096e90 2024-04-27T05:21:11.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.824+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3941266e0 con 0x7fc394096e90 2024-04-27T05:21:11.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.824+0000 7fc3917fa640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc378037700 0x7fc378039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:11.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.824+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fc38806bdc0 con 0x7fc394096e90 2024-04-27T05:21:11.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.824+0000 7fc393fff640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc378037700 0x7fc378039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:11.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.825+0000 7fc393fff640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc378037700 0x7fc378039bc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc380000c00 tx=0x7fc380007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:11.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.828+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fc38803a740 con 0x7fc394096e90 2024-04-27T05:21:11.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.969+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f41b0061a20 con 0x7f41b0097990 2024-04-27T05:21:11.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.970+0000 7f419effd640 1 -- 172.21.15.114:0/1464293775 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f41a0018070 con 0x7f41b0097990 2024-04-27T05:21:11.972 INFO:teuthology.orchestra.run.smithi114.stdout:47244640262 2024-04-27T05:21:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f41900376b0 msgr2=0x7f4190039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f41900376b0 0x7f4190039b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41a4036000 tx=0x7f41a4008030 comp rx=0 tx=0).stop 2024-04-27T05:21:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 msgr2=0x7f41b012f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 0x7f41b012f390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41a000c420 tx=0x7f41a000c8e0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 shutdown_connections 2024-04-27T05:21:11.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f41900376b0 0x7f4190039b70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 --2- 172.21.15.114:0/1464293775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f41b0097990 0x7f41b012f390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 >> 172.21.15.114:0/1464293775 conn(0x7f41b009fab0 msgr2=0x7f41b009ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:11.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 shutdown_connections 2024-04-27T05:21:11.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.973+0000 7f41b4e62640 1 -- 172.21.15.114:0/1464293775 wait complete. 2024-04-27T05:21:11.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.2 2024-04-27T05:21:11.986 DEBUG:teuthology.parallel:result is None 2024-04-27T05:21:11.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.992+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f95e4071fa0 con 0x7f95e4096e90 2024-04-27T05:21:11.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.993+0000 7f95cb7fe640 1 -- 172.21.15.114:0/1283269107 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95d4019cd0 con 0x7f95e4096e90 2024-04-27T05:21:11.995 INFO:teuthology.orchestra.run.smithi114.stdout:47244640262 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95d0037700 msgr2=0x7f95d0039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95d0037700 0x7f95d0039bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097f0 tx=0x7f95d8008030 comp rx=0 tx=0).stop 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 msgr2=0x7f95e4070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 0x7f95e4070340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f95d400c340 tx=0x7f95d400c800 comp rx=0 tx=0).stop 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 shutdown_connections 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f95d0037700 0x7f95d0039bc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 --2- 172.21.15.114:0/1283269107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95e4096e90 0x7f95e4070340 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:11.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 >> 172.21.15.114:0/1283269107 conn(0x7f95e409d700 msgr2=0x7f95e4131e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:11.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 shutdown_connections 2024-04-27T05:21:11.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:11.995+0000 7f95e9444640 1 -- 172.21.15.114:0/1283269107 wait complete. 2024-04-27T05:21:12.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.3 2024-04-27T05:21:12.007 DEBUG:teuthology.parallel:result is None 2024-04-27T05:21:12.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.008+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc39406c710 con 0x7fc394096e90 2024-04-27T05:21:12.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.008+0000 7fc3917fa640 1 -- 172.21.15.114:0/1758362369 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc38803a380 con 0x7fc394096e90 2024-04-27T05:21:12.010 INFO:teuthology.orchestra.run.smithi114.stdout:38654705670 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc378037700 msgr2=0x7fc378039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc378037700 0x7fc378039bc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc380000c00 tx=0x7fc380007970 comp rx=0 tx=0).stop 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 msgr2=0x7fc3941c17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc3941c17c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3880316a0 tx=0x7fc3880046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 shutdown_connections 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fc378037700 0x7fc378039bc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 --2- 172.21.15.114:0/1758362369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc394096e90 0x7fc3941c17c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 >> 172.21.15.114:0/1758362369 conn(0x7fc39409d700 msgr2=0x7fc394131e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 shutdown_connections 2024-04-27T05:21:12.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.010+0000 7fc39adc8640 1 -- 172.21.15.114:0/1758362369 wait complete. 2024-04-27T05:21:12.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.016+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22f4061a20 con 0x7f22f4097990 2024-04-27T05:21:12.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.016+0000 7f22e9ffb640 1 -- 172.21.15.114:0/3461900367 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22f0017030 con 0x7f22f4097990 2024-04-27T05:21:12.018 INFO:teuthology.orchestra.run.smithi114.stdout:38654705669 2024-04-27T05:21:12.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.018+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f22d8037700 msgr2=0x7f22d8039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.018+0000 7f22fb2e9640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f22d8037700 0x7f22d8039bc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22e4036000 tx=0x7f22e4008030 comp rx=0 tx=0).stop 2024-04-27T05:21:12.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.018+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 msgr2=0x7f22f412f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.018+0000 7f22fb2e9640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 0x7f22f412f800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c450 tx=0x7f22f000c910 comp rx=0 tx=0).stop 2024-04-27T05:21:12.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.019+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 shutdown_connections 2024-04-27T05:21:12.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.019+0000 7f22fb2e9640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f22d8037700 0x7f22d8039bc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.019+0000 7f22fb2e9640 1 --2- 172.21.15.114:0/3461900367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22f4097990 0x7f22f412f800 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.019+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 >> 172.21.15.114:0/3461900367 conn(0x7f22f409fab0 msgr2=0x7f22f409b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.019+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 shutdown_connections 2024-04-27T05:21:12.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.019+0000 7f22fb2e9640 1 -- 172.21.15.114:0/3461900367 wait complete. 2024-04-27T05:21:12.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-04-27T05:21:12.023 DEBUG:teuthology.parallel:result is None 2024-04-27T05:21:12.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-04-27T05:21:12.031 DEBUG:teuthology.parallel:result is None 2024-04-27T05:21:12.031 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T05:21:12.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.164+0000 7ffb2bb3c640 1 Processor -- start 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.164+0000 7ffb2bb3c640 1 -- start start 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb2bb3c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb2412ff90 0x7ffb24132380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb2bb3c640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb24099270 con 0x7ffb24095880 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb2bb3c640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb24098c60 con 0x7ffb2412ff90 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb290d7640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb2412ff90 0x7ffb24132380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb298d8640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ffb24095880 0x7ffb24095c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53130/0 (socket says 172.21.15.114:53130) 2024-04-27T05:21:12.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb298d8640 1 -- 172.21.15.114:0/11860180 learned_addr learned my addr 172.21.15.114:0/11860180 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:12.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb290d7640 1 -- 172.21.15.114:0/11860180 >> v1:172.21.15.114:6789/0 conn(0x7ffb24095880 legacy=0x7ffb24095c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb288d6640 1 -- 172.21.15.114:0/11860180 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853988252 0 0) 0x7ffb24099270 con 0x7ffb24095880 2024-04-27T05:21:12.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.165+0000 7ffb290d7640 1 -- 172.21.15.114:0/11860180 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb24096370 con 0x7ffb2412ff90 2024-04-27T05:21:12.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.166+0000 7ffb290d7640 1 --2- 172.21.15.114:0/11860180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb2412ff90 0x7ffb24132380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009750 tx=0x7ffb1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c15904467482e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.166+0000 7ffb288d6640 1 -- 172.21.15.114:0/11860180 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb14031760 con 0x7ffb2412ff90 2024-04-27T05:21:12.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.166+0000 7ffb288d6640 1 -- 172.21.15.114:0/11860180 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb140318c0 con 0x7ffb2412ff90 2024-04-27T05:21:12.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.166+0000 7ffb288d6640 1 -- 172.21.15.114:0/11860180 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb14031b80 con 0x7ffb2412ff90 2024-04-27T05:21:12.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.166+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/11860180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb2412ff90 msgr2=0x7ffb24132380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.166+0000 7ffb2bb3c640 1 --2- 172.21.15.114:0/11860180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb2412ff90 0x7ffb24132380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009750 tx=0x7ffb1402b8d0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/11860180 shutdown_connections 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 --2- 172.21.15.114:0/11860180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb2412ff90 0x7ffb24132380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/11860180 >> 172.21.15.114:0/11860180 conn(0x7ffb2409f960 msgr2=0x7ffb2409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/11860180 shutdown_connections 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/11860180 wait complete. 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 Processor -- start 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.167+0000 7ffb2bb3c640 1 -- start start 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb2bb3c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 0x7ffb2406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb2bb3c640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb24071770 con 0x7ffb24095880 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb298d8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 0x7ffb2406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb298d8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 0x7ffb2406e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49956/0 (socket says 172.21.15.114:49956) 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb298d8640 1 -- 172.21.15.114:0/1281101039 learned_addr learned my addr 172.21.15.114:0/1281101039 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb298d8640 1 -- 172.21.15.114:0/1281101039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14009400 con 0x7ffb24095880 2024-04-27T05:21:12.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb298d8640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 0x7ffb2406e3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c420 tx=0x7ffb2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb200074d0 con 0x7ffb24095880 2024-04-27T05:21:12.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20007630 con 0x7ffb24095880 2024-04-27T05:21:12.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb240709a0 con 0x7ffb24095880 2024-04-27T05:21:12.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb20004950 con 0x7ffb24095880 2024-04-27T05:21:12.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.168+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb24124600 con 0x7ffb24095880 2024-04-27T05:21:12.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.169+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7ffb20004b70 con 0x7ffb24095880 2024-04-27T05:21:12.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.169+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb24061a20 con 0x7ffb24095880 2024-04-27T05:21:12.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.169+0000 7ffb1a7fc640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7ffb080376b0 0x7ffb08039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.169+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7ffb200495a0 con 0x7ffb24095880 2024-04-27T05:21:12.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.170+0000 7ffb290d7640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7ffb080376b0 0x7ffb08039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.171+0000 7ffb290d7640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7ffb080376b0 0x7ffb08039b70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb14036000 tx=0x7ffb14008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.174+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7ffb2001ab40 con 0x7ffb24095880 2024-04-27T05:21:12.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.338+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 --> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb240994a0 con 0x7ffb080376b0 2024-04-27T05:21:12.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.340+0000 7ffb1a7fc640 1 -- 172.21.15.114:0/1281101039 <== mgr.4101 v2:172.21.15.138:6824/29568 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34613 (secure 0 0 0) 0x7ffb14031b30 con 0x7ffb080376b0 2024-04-27T05:21:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:21:12.343 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-04-27T05:21:12.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7ffb080376b0 msgr2=0x7ffb08039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7ffb080376b0 0x7ffb08039b70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb14036000 tx=0x7ffb14008030 comp rx=0 tx=0).stop 2024-04-27T05:21:12.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 msgr2=0x7ffb2406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 0x7ffb2406e3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c420 tx=0x7ffb2000c8e0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 shutdown_connections 2024-04-27T05:21:12.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7ffb080376b0 0x7ffb08039b70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 --2- 172.21.15.114:0/1281101039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb24095880 0x7ffb2406e3f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 >> 172.21.15.114:0/1281101039 conn(0x7ffb2409f960 msgr2=0x7ffb24131b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 shutdown_connections 2024-04-27T05:21:12.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.342+0000 7ffb2bb3c640 1 -- 172.21.15.114:0/1281101039 wait complete. 2024-04-27T05:21:12.354 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-27T05:21:10.791698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":109756,"kb_used_data":1672,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377604,"statfs":{"total":386547056640,"available":386434666496,"internally_reserved":0,"allocated":1712128,"data_stored":1340100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6373,"internal_metadata":110618395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001076"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:08.657613+0000","last_change":"2024-04-27T05:21:08.655789+0000","last_active":"2024-04-27T05:21:08.657613+0000","last_peered":"2024-04-27T05:21:08.657613+0000","last_clean":"2024-04-27T05:21:08.657613+0000","last_became_active":"2024-04-27T05:21:08.655464+0000","last_became_peered":"2024-04-27T05:21:08.655464+0000","last_unstale":"2024-04-27T05:21:08.657613+0000","last_undegraded":"2024-04-27T05:21:08.657613+0000","last_fullsized":"2024-04-27T05:21:08.657613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:44.891873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.878699+0000","last_change":"2024-04-27T05:21:07.878156+0000","last_active":"2024-04-27T05:21:07.878699+0000","last_peered":"2024-04-27T05:21:07.878699+0000","last_clean":"2024-04-27T05:21:07.878699+0000","last_became_active":"2024-04-27T05:21:07.877920+0000","last_became_peered":"2024-04-27T05:21:07.877920+0000","last_unstale":"2024-04-27T05:21:07.878699+0000","last_undegraded":"2024-04-27T05:21:07.878699+0000","last_fullsized":"2024-04-27T05:21:07.878699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:19:52.511869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.877495+0000","last_change":"2024-04-27T05:21:07.876370+0000","last_active":"2024-04-27T05:21:07.877495+0000","last_peered":"2024-04-27T05:21:07.877495+0000","last_clean":"2024-04-27T05:21:07.877495+0000","last_became_active":"2024-04-27T05:21:07.876126+0000","last_became_peered":"2024-04-27T05:21:07.876126+0000","last_unstale":"2024-04-27T05:21:07.877495+0000","last_undegraded":"2024-04-27T05:21:07.877495+0000","last_fullsized":"2024-04-27T05:21:07.877495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:59:25.984400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.878433+0000","last_change":"2024-04-27T05:21:07.877762+0000","last_active":"2024-04-27T05:21:07.878433+0000","last_peered":"2024-04-27T05:21:07.878433+0000","last_clean":"2024-04-27T05:21:07.878433+0000","last_became_active":"2024-04-27T05:21:07.877535+0000","last_became_peered":"2024-04-27T05:21:07.877535+0000","last_unstale":"2024-04-27T05:21:07.878433+0000","last_undegraded":"2024-04-27T05:21:07.878433+0000","last_fullsized":"2024-04-27T05:21:07.878433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:56:08.133862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.878871+0000","last_change":"2024-04-27T05:21:07.877483+0000","last_active":"2024-04-27T05:21:07.878871+0000","last_peered":"2024-04-27T05:21:07.878871+0000","last_clean":"2024-04-27T05:21:07.878871+0000","last_became_active":"2024-04-27T05:21:07.877215+0000","last_became_peered":"2024-04-27T05:21:07.877215+0000","last_unstale":"2024-04-27T05:21:07.878871+0000","last_undegraded":"2024-04-27T05:21:07.878871+0000","last_fullsized":"2024-04-27T05:21:07.878871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:04:20.366774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.877618+0000","last_change":"2024-04-27T05:21:07.875880+0000","last_active":"2024-04-27T05:21:07.877618+0000","last_peered":"2024-04-27T05:21:07.877618+0000","last_clean":"2024-04-27T05:21:07.877618+0000","last_became_active":"2024-04-27T05:21:07.875615+0000","last_became_peered":"2024-04-27T05:21:07.875615+0000","last_unstale":"2024-04-27T05:21:07.877618+0000","last_undegraded":"2024-04-27T05:21:07.877618+0000","last_fullsized":"2024-04-27T05:21:07.877618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:30.324314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.877551+0000","last_change":"2024-04-27T05:21:07.876465+0000","last_active":"2024-04-27T05:21:07.877551+0000","last_peered":"2024-04-27T05:21:07.877551+0000","last_clean":"2024-04-27T05:21:07.877551+0000","last_became_active":"2024-04-27T05:21:07.876207+0000","last_became_peered":"2024-04-27T05:21:07.876207+0000","last_unstale":"2024-04-27T05:21:07.877551+0000","last_undegraded":"2024-04-27T05:21:07.877551+0000","last_fullsized":"2024-04-27T05:21:07.877551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:48:31.903969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.879596+0000","last_change":"2024-04-27T05:21:07.878323+0000","last_active":"2024-04-27T05:21:07.879596+0000","last_peered":"2024-04-27T05:21:07.879596+0000","last_clean":"2024-04-27T05:21:07.879596+0000","last_became_active":"2024-04-27T05:21:07.877956+0000","last_became_peered":"2024-04-27T05:21:07.877956+0000","last_unstale":"2024-04-27T05:21:07.879596+0000","last_undegraded":"2024-04-27T05:21:07.879596+0000","last_fullsized":"2024-04-27T05:21:07.879596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:43.362463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":164,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-27T05:21:06.872827+0000","last_change":"2024-04-27T05:21:00.911533+0000","last_active":"2024-04-27T05:21:06.872827+0000","last_peered":"2024-04-27T05:21:06.872827+0000","last_clean":"2024-04-27T05:21:06.872827+0000","last_became_active":"2024-04-27T05:21:00.910548+0000","last_became_peered":"2024-04-27T05:21:00.910548+0000","last_unstale":"2024-04-27T05:21:06.872827+0000","last_undegraded":"2024-04-27T05:21:06.872827+0000","last_fullsized":"2024-04-27T05:21:06.872827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:20:58.849194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:20:58.849194+0000","last_clean_scrub_stamp":"2024-04-27T05:20:58.849194+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:09.200756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":39841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":630209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":630209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":39841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T05:21:12.355 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.447+0000 7f7547b86640 1 Processor -- start 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7547b86640 1 -- start start 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7547b86640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f75400983a0 0x7f754007e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7547b86640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75400953d0 con 0x7f7540093430 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7547b86640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75400842f0 con 0x7f75400983a0 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7545121640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f75400983a0 0x7f754007e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7545922640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7540093430 0x7f7540093810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53140/0 (socket says 172.21.15.114:53140) 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7545922640 1 -- 172.21.15.114:0/2506734386 learned_addr learned my addr 172.21.15.114:0/2506734386 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:12.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.448+0000 7f7544920640 1 -- 172.21.15.114:0/2506734386 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204940269 0 0) 0x7f75400953d0 con 0x7f7540093430 2024-04-27T05:21:12.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.465+0000 7f7544920640 1 -- 172.21.15.114:0/2506734386 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c003500 con 0x7f7540093430 2024-04-27T05:21:12.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.465+0000 7f7545121640 1 -- 172.21.15.114:0/2506734386 >> v1:172.21.15.114:6789/0 conn(0x7f7540093430 legacy=0x7f7540093810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.465+0000 7f7545121640 1 -- 172.21.15.114:0/2506734386 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400950c0 con 0x7f75400983a0 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7545121640 1 --2- 172.21.15.114:0/2506734386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75400983a0 0x7f754007e200 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f753c009720 tx=0x7f753c02ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985377f6be867a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7544920640 1 -- 172.21.15.114:0/2506734386 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f753c02d750 con 0x7f75400983a0 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7544920640 1 -- 172.21.15.114:0/2506734386 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c02d8b0 con 0x7f75400983a0 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7544920640 1 -- 172.21.15.114:0/2506734386 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f753c02db70 con 0x7f75400983a0 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 -- 172.21.15.114:0/2506734386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75400983a0 msgr2=0x7f754007e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 --2- 172.21.15.114:0/2506734386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75400983a0 0x7f754007e200 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f753c009720 tx=0x7f753c02ba60 comp rx=0 tx=0).stop 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 -- 172.21.15.114:0/2506734386 shutdown_connections 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 --2- 172.21.15.114:0/2506734386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75400983a0 0x7f754007e200 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 -- 172.21.15.114:0/2506734386 >> 172.21.15.114:0/2506734386 conn(0x7f754009b550 msgr2=0x7f7540097a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 -- 172.21.15.114:0/2506734386 shutdown_connections 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.466+0000 7f7547b86640 1 -- 172.21.15.114:0/2506734386 wait complete. 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.467+0000 7f7547b86640 1 Processor -- start 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.467+0000 7f7547b86640 1 -- start start 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7547b86640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 0x7f754008b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7547b86640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754008c5e0 con 0x7f7540093430 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7545922640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 0x7f754008b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7545922640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 0x7f754008b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49970/0 (socket says 172.21.15.114:49970) 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7545922640 1 -- 172.21.15.114:0/266465846 learned_addr learned my addr 172.21.15.114:0/266465846 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7545922640 1 -- 172.21.15.114:0/266465846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0091e0 con 0x7f7540093430 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7545922640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 0x7f754008b4c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f753000c480 tx=0x7f753000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7530005c10 con 0x7f7540093430 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7530005d70 con 0x7f7540093430 2024-04-27T05:21:12.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7540089e80 con 0x7f7540093430 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75300071e0 con 0x7f7540093430 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.468+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7540089460 con 0x7f7540093430 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.469+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f7530007340 con 0x7f7540093430 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.469+0000 7f75367fc640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7524037700 0x7f7524039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.469+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7540061a20 con 0x7f7540093430 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.469+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f75300494e0 con 0x7f7540093430 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.469+0000 7f7545121640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7524037700 0x7f7524039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.470+0000 7f7545121640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7524037700 0x7f7524039bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f753c036000 tx=0x7f753c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.474+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f7530017070 con 0x7f7540093430 2024-04-27T05:21:12.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.641+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 --> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7540094c80 con 0x7f7524037700 2024-04-27T05:21:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.643+0000 7f75367fc640 1 -- 172.21.15.114:0/266465846 <== mgr.4101 v2:172.21.15.138:6824/29568 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34613 (secure 0 0 0) 0x7f753c03d040 con 0x7f7524037700 2024-04-27T05:21:12.645 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:21:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7524037700 msgr2=0x7f7524039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7524037700 0x7f7524039bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f753c036000 tx=0x7f753c008030 comp rx=0 tx=0).stop 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 msgr2=0x7f754008b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 0x7f754008b4c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f753000c480 tx=0x7f753000c940 comp rx=0 tx=0).stop 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 shutdown_connections 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7524037700 0x7f7524039bc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 --2- 172.21.15.114:0/266465846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7540093430 0x7f754008b4c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 >> 172.21.15.114:0/266465846 conn(0x7f754009b550 msgr2=0x7f7540096210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 shutdown_connections 2024-04-27T05:21:12.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.645+0000 7f7547b86640 1 -- 172.21.15.114:0/266465846 wait complete. 2024-04-27T05:21:12.657 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-27T05:21:10.791698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":109756,"kb_used_data":1672,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377604,"statfs":{"total":386547056640,"available":386434666496,"internally_reserved":0,"allocated":1712128,"data_stored":1340100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6373,"internal_metadata":110618395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001076"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:08.657613+0000","last_change":"2024-04-27T05:21:08.655789+0000","last_active":"2024-04-27T05:21:08.657613+0000","last_peered":"2024-04-27T05:21:08.657613+0000","last_clean":"2024-04-27T05:21:08.657613+0000","last_became_active":"2024-04-27T05:21:08.655464+0000","last_became_peered":"2024-04-27T05:21:08.655464+0000","last_unstale":"2024-04-27T05:21:08.657613+0000","last_undegraded":"2024-04-27T05:21:08.657613+0000","last_fullsized":"2024-04-27T05:21:08.657613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:44.891873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.878699+0000","last_change":"2024-04-27T05:21:07.878156+0000","last_active":"2024-04-27T05:21:07.878699+0000","last_peered":"2024-04-27T05:21:07.878699+0000","last_clean":"2024-04-27T05:21:07.878699+0000","last_became_active":"2024-04-27T05:21:07.877920+0000","last_became_peered":"2024-04-27T05:21:07.877920+0000","last_unstale":"2024-04-27T05:21:07.878699+0000","last_undegraded":"2024-04-27T05:21:07.878699+0000","last_fullsized":"2024-04-27T05:21:07.878699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:19:52.511869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.877495+0000","last_change":"2024-04-27T05:21:07.876370+0000","last_active":"2024-04-27T05:21:07.877495+0000","last_peered":"2024-04-27T05:21:07.877495+0000","last_clean":"2024-04-27T05:21:07.877495+0000","last_became_active":"2024-04-27T05:21:07.876126+0000","last_became_peered":"2024-04-27T05:21:07.876126+0000","last_unstale":"2024-04-27T05:21:07.877495+0000","last_undegraded":"2024-04-27T05:21:07.877495+0000","last_fullsized":"2024-04-27T05:21:07.877495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:59:25.984400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.878433+0000","last_change":"2024-04-27T05:21:07.877762+0000","last_active":"2024-04-27T05:21:07.878433+0000","last_peered":"2024-04-27T05:21:07.878433+0000","last_clean":"2024-04-27T05:21:07.878433+0000","last_became_active":"2024-04-27T05:21:07.877535+0000","last_became_peered":"2024-04-27T05:21:07.877535+0000","last_unstale":"2024-04-27T05:21:07.878433+0000","last_undegraded":"2024-04-27T05:21:07.878433+0000","last_fullsized":"2024-04-27T05:21:07.878433+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:56:08.133862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.878871+0000","last_change":"2024-04-27T05:21:07.877483+0000","last_active":"2024-04-27T05:21:07.878871+0000","last_peered":"2024-04-27T05:21:07.878871+0000","last_clean":"2024-04-27T05:21:07.878871+0000","last_became_active":"2024-04-27T05:21:07.877215+0000","last_became_peered":"2024-04-27T05:21:07.877215+0000","last_unstale":"2024-04-27T05:21:07.878871+0000","last_undegraded":"2024-04-27T05:21:07.878871+0000","last_fullsized":"2024-04-27T05:21:07.878871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:04:20.366774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.877618+0000","last_change":"2024-04-27T05:21:07.875880+0000","last_active":"2024-04-27T05:21:07.877618+0000","last_peered":"2024-04-27T05:21:07.877618+0000","last_clean":"2024-04-27T05:21:07.877618+0000","last_became_active":"2024-04-27T05:21:07.875615+0000","last_became_peered":"2024-04-27T05:21:07.875615+0000","last_unstale":"2024-04-27T05:21:07.877618+0000","last_undegraded":"2024-04-27T05:21:07.877618+0000","last_fullsized":"2024-04-27T05:21:07.877618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:30.324314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.877551+0000","last_change":"2024-04-27T05:21:07.876465+0000","last_active":"2024-04-27T05:21:07.877551+0000","last_peered":"2024-04-27T05:21:07.877551+0000","last_clean":"2024-04-27T05:21:07.877551+0000","last_became_active":"2024-04-27T05:21:07.876207+0000","last_became_peered":"2024-04-27T05:21:07.876207+0000","last_unstale":"2024-04-27T05:21:07.877551+0000","last_undegraded":"2024-04-27T05:21:07.877551+0000","last_fullsized":"2024-04-27T05:21:07.877551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:48:31.903969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-27T05:21:07.879596+0000","last_change":"2024-04-27T05:21:07.878323+0000","last_active":"2024-04-27T05:21:07.879596+0000","last_peered":"2024-04-27T05:21:07.879596+0000","last_clean":"2024-04-27T05:21:07.879596+0000","last_became_active":"2024-04-27T05:21:07.877956+0000","last_became_peered":"2024-04-27T05:21:07.877956+0000","last_unstale":"2024-04-27T05:21:07.879596+0000","last_undegraded":"2024-04-27T05:21:07.879596+0000","last_fullsized":"2024-04-27T05:21:07.879596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:21:06.865122+0000","last_clean_scrub_stamp":"2024-04-27T05:21:06.865122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:43.362463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":164,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-27T05:21:06.872827+0000","last_change":"2024-04-27T05:21:00.911533+0000","last_active":"2024-04-27T05:21:06.872827+0000","last_peered":"2024-04-27T05:21:06.872827+0000","last_clean":"2024-04-27T05:21:06.872827+0000","last_became_active":"2024-04-27T05:21:00.910548+0000","last_became_peered":"2024-04-27T05:21:00.910548+0000","last_unstale":"2024-04-27T05:21:06.872827+0000","last_undegraded":"2024-04-27T05:21:06.872827+0000","last_fullsized":"2024-04-27T05:21:06.872827+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T05:20:58.849194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T05:20:58.849194+0000","last_clean_scrub_stamp":"2024-04-27T05:20:58.849194+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:09.200756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":39841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":630209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]}]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":630209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":131072,"data_stored":39841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T05:21:12.658 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T05:21:12.658 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-27T05:21:12.658 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-27T05:21:12.659 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-27T05:21:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f7279c3a640 1 Processor -- start 2024-04-27T05:21:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f7279c3a640 1 -- start start 2024-04-27T05:21:12.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f7279c3a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7274127bf0 0x7f7274127fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f7279c3a640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72740903d0 con 0x7f727412ac30 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f7279c3a640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7274090c30 con 0x7f7274127bf0 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f72737fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7274127bf0 0x7f7274127fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f726bfff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f727412ac30 0x7f727412b0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:53156/0 (socket says 172.21.15.114:53156) 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f726bfff640 1 -- 172.21.15.114:0/960916699 learned_addr learned my addr 172.21.15.114:0/960916699 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.751+0000 7f72737fe640 1 --2- 172.21.15.114:0/960916699 >> v2:172.21.15.114:3300/0 conn(0x7f7274127bf0 0x7f7274127fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49972/0 (socket says 172.21.15.114:49972) 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.752+0000 7f72737fe640 1 -- 172.21.15.114:0/960916699 >> v1:172.21.15.114:6789/0 conn(0x7f727412ac30 legacy=0x7f727412b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.752+0000 7f72737fe640 1 -- 172.21.15.114:0/960916699 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274127730 con 0x7f7274127bf0 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.752+0000 7f72737fe640 1 --2- 172.21.15.114:0/960916699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274127fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7264009720 tx=0x7f726402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7299c6bd51282913 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.752+0000 7f7272ffd640 1 -- 172.21.15.114:0/960916699 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7264031760 con 0x7f7274127bf0 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.752+0000 7f7272ffd640 1 -- 172.21.15.114:0/960916699 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72640318c0 con 0x7f7274127bf0 2024-04-27T05:21:12.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.752+0000 7f7272ffd640 1 -- 172.21.15.114:0/960916699 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7264031b80 con 0x7f7274127bf0 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 -- 172.21.15.114:0/960916699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 msgr2=0x7f7274127fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 --2- 172.21.15.114:0/960916699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274127fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7264009720 tx=0x7f726402b8d0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 -- 172.21.15.114:0/960916699 shutdown_connections 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 --2- 172.21.15.114:0/960916699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274127fd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 -- 172.21.15.114:0/960916699 >> 172.21.15.114:0/960916699 conn(0x7f7274098220 msgr2=0x7f7274095a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 -- 172.21.15.114:0/960916699 shutdown_connections 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.753+0000 7f7279c3a640 1 -- 172.21.15.114:0/960916699 wait complete. 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f7279c3a640 1 Processor -- start 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f7279c3a640 1 -- start start 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f7279c3a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274124540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f7279c3a640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7274126f70 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f72737fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274124540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f72737fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274124540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49978/0 (socket says 172.21.15.114:49978) 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f72737fe640 1 -- 172.21.15.114:0/370346768 learned_addr learned my addr 172.21.15.114:0/370346768 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.754+0000 7f72737fe640 1 -- 172.21.15.114:0/370346768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7264009400 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f72737fe640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274124540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7264009720 tx=0x7f72640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f726403a860 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726403a9e0 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72740891b0 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f726403acc0 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7274126b40 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.755+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f726404d450 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.756+0000 7f7270ff9640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7254037700 0x7f7254039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.756+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f726406c190 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.756+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72741271e0 con 0x7f7274127bf0 2024-04-27T05:21:12.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.756+0000 7f726bfff640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7254037700 0x7f7254039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:12.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.757+0000 7f726bfff640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7254037700 0x7f7254039bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f725c007fd0 tx=0x7f725c007ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:12.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.760+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f726404b350 con 0x7f7274127bf0 2024-04-27T05:21:12.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.967+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7274095090 con 0x7f7274127bf0 2024-04-27T05:21:12.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.968+0000 7f7270ff9640 1 -- 172.21.15.114:0/370346768 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7274095090 con 0x7f7274127bf0 2024-04-27T05:21:12.970 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-27T05:21:12.970 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-27T05:21:12.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.969+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7254037700 msgr2=0x7f7254039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.969+0000 7f7279c3a640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7254037700 0x7f7254039bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f725c007fd0 tx=0x7f725c007ac0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.969+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 msgr2=0x7f7274124540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.969+0000 7f7279c3a640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274124540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7264009720 tx=0x7f72640046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.970+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 shutdown_connections 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.970+0000 7f7279c3a640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f7254037700 0x7f7254039bc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.970+0000 7f7279c3a640 1 --2- 172.21.15.114:0/370346768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7274127bf0 0x7f7274124540 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.970+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 >> 172.21.15.114:0/370346768 conn(0x7f7274098220 msgr2=0x7f7274128a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.970+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 shutdown_connections 2024-04-27T05:21:12.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:21:12.970+0000 7f7279c3a640 1 -- 172.21.15.114:0/370346768 wait complete. 2024-04-27T05:21:12.982 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-27T05:21:12.982 INFO:teuthology.run_tasks:Running task rgw... 2024-04-27T05:21:12.997 DEBUG:tasks.rgw:config is {'client.0': None} 2024-04-27T05:21:12.997 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-04-27T05:21:12.997 INFO:tasks.rgw:Creating data pools 2024-04-27T05:21:12.998 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-04-27T05:21:12.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.098+0000 7fe8ce48e640 1 Processor -- start 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.098+0000 7fe8ce48e640 1 -- start start 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8ce48e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe8c81172b0 0x7fe8c81196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8ce48e640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c8119c70 con 0x7fe8c8114570 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8ce48e640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c811ae10 con 0x7fe8c81172b0 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c77fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe8c81172b0 0x7fe8c81196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c77fe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe8c81172b0 0x7fe8c81196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36308/0 (socket says 172.21.15.138:36308) 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c77fe640 1 -- 172.21.15.138:0/73168272 learned_addr learned my addr 172.21.15.138:0/73168272 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c7fff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe8c8114570 0x7fe8c8114950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38278/0 (socket says 172.21.15.138:38278) 2024-04-27T05:21:13.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c77fe640 1 -- 172.21.15.138:0/73168272 >> v1:172.21.15.114:6789/0 conn(0x7fe8c8114570 legacy=0x7fe8c8114950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:13.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c6ffd640 1 -- 172.21.15.138:0/73168272 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808778516 0 0) 0x7fe8c8119c70 con 0x7fe8c8114570 2024-04-27T05:21:13.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.099+0000 7fe8c77fe640 1 -- 172.21.15.138:0/73168272 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c811af80 con 0x7fe8c81172b0 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.100+0000 7fe8c77fe640 1 --2- 172.21.15.138:0/73168272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c81172b0 0x7fe8c81196a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0097f0 tx=0x7fe8bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eebdc03a20779 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.100+0000 7fe8c6ffd640 1 -- 172.21.15.138:0/73168272 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8bc0317a0 con 0x7fe8c81172b0 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.100+0000 7fe8c6ffd640 1 -- 172.21.15.138:0/73168272 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc031900 con 0x7fe8c81172b0 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.100+0000 7fe8c6ffd640 1 -- 172.21.15.138:0/73168272 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8bc031bc0 con 0x7fe8c81172b0 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 -- 172.21.15.138:0/73168272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c81172b0 msgr2=0x7fe8c81196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 --2- 172.21.15.138:0/73168272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c81172b0 0x7fe8c81196a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0097f0 tx=0x7fe8bc02b910 comp rx=0 tx=0).stop 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 -- 172.21.15.138:0/73168272 shutdown_connections 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 --2- 172.21.15.138:0/73168272 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c81172b0 0x7fe8c81196a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 -- 172.21.15.138:0/73168272 >> 172.21.15.138:0/73168272 conn(0x7fe8c8071b40 msgr2=0x7fe8c806be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:13.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 -- 172.21.15.138:0/73168272 shutdown_connections 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 -- 172.21.15.138:0/73168272 wait complete. 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.101+0000 7fe8ce48e640 1 Processor -- start 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8ce48e640 1 -- start start 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8ce48e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 0x7fe8c81a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8ce48e640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81a3180 con 0x7fe8c8114570 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 0x7fe8c81a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c7fff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 0x7fe8c81a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36314/0 (socket says 172.21.15.138:36314) 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c7fff640 1 -- 172.21.15.138:0/4114889287 learned_addr learned my addr 172.21.15.138:0/4114889287 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c7fff640 1 -- 172.21.15.138:0/4114889287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0094d0 con 0x7fe8c8114570 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c7fff640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 0x7fe8c81a2bb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c420 tx=0x7fe8b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8b80074d0 con 0x7fe8c8114570 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8007630 con 0x7fe8c8114570 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c81a33e0 con 0x7fe8c8114570 2024-04-27T05:21:13.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8b8004950 con 0x7fe8c8114570 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.102+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81a7480 con 0x7fe8c8114570 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.103+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fe8b8004b70 con 0x7fe8c8114570 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.103+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c804e9b0 con 0x7fe8c8114570 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.103+0000 7fe8c4ff9640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe8b4037700 0x7fe8b4039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.103+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fe8b80496a0 con 0x7fe8c8114570 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.104+0000 7fe8c77fe640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe8b4037700 0x7fe8b4039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:13.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.104+0000 7fe8c77fe640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe8b4037700 0x7fe8b4039bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc036000 tx=0x7fe8bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:13.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.108+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fe8b801ab40 con 0x7fe8c8114570 2024-04-27T05:21:13.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.278+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7fe8c81a35d0 con 0x7fe8c8114570 2024-04-27T05:21:13.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.887+0000 7fe8c4ff9640 1 -- 172.21.15.138:0/4114889287 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.data' created v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe8b8018030 con 0x7fe8c8114570 2024-04-27T05:21:13.888 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'default.rgw.buckets.data' created 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.891+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe8b4037700 msgr2=0x7fe8b4039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.891+0000 7fe8ce48e640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe8b4037700 0x7fe8b4039bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc036000 tx=0x7fe8bc008030 comp rx=0 tx=0).stop 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.891+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 msgr2=0x7fe8c81a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.891+0000 7fe8ce48e640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 0x7fe8c81a2bb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c420 tx=0x7fe8b800c8e0 comp rx=0 tx=0).stop 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.892+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 shutdown_connections 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.892+0000 7fe8ce48e640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fe8b4037700 0x7fe8b4039bc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:13.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.892+0000 7fe8ce48e640 1 --2- 172.21.15.138:0/4114889287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8c8114570 0x7fe8c81a2bb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:13.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.892+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 >> 172.21.15.138:0/4114889287 conn(0x7fe8c8071b40 msgr2=0x7fe8c81193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:13.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.892+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 shutdown_connections 2024-04-27T05:21:13.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:13.892+0000 7fe8ce48e640 1 -- 172.21.15.138:0/4114889287 wait complete. 2024-04-27T05:21:13.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-04-27T05:21:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd91b08f640 1 Processor -- start 2024-04-27T05:21:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd91b08f640 1 -- start start 2024-04-27T05:21:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd91b08f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd914117220 0x7fd914119610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd91b08f640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd914119b50 con 0x7fd914114410 2024-04-27T05:21:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd91b08f640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91411acf0 con 0x7fd914117220 2024-04-27T05:21:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd913fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd914117220 0x7fd914119610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd918e2b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fd914114410 0x7fd9141147f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38282/0 (socket says 172.21.15.138:38282) 2024-04-27T05:21:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd918e2b640 1 -- 172.21.15.138:0/1557631445 learned_addr learned my addr 172.21.15.138:0/1557631445 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.012+0000 7fd913fff640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd914117220 0x7fd914119610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36316/0 (socket says 172.21.15.138:36316) 2024-04-27T05:21:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.013+0000 7fd913fff640 1 -- 172.21.15.138:0/1557631445 >> v1:172.21.15.114:6789/0 conn(0x7fd914114410 legacy=0x7fd9141147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.013+0000 7fd9137fe640 1 -- 172.21.15.138:0/1557631445 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643345961 0 0) 0x7fd914119b50 con 0x7fd914114410 2024-04-27T05:21:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.013+0000 7fd913fff640 1 -- 172.21.15.138:0/1557631445 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91411ae30 con 0x7fd914117220 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.013+0000 7fd913fff640 1 --2- 172.21.15.138:0/1557631445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914117220 0x7fd914119610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd900009720 tx=0x7fd90002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2b3a68c1651c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd9137fe640 1 -- 172.21.15.138:0/1557631445 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd90002e070 con 0x7fd914117220 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd9137fe640 1 -- 172.21.15.138:0/1557631445 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9000316d0 con 0x7fd914117220 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd9137fe640 1 -- 172.21.15.138:0/1557631445 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd900031990 con 0x7fd914117220 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 -- 172.21.15.138:0/1557631445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914117220 msgr2=0x7fd914119610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 --2- 172.21.15.138:0/1557631445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914117220 0x7fd914119610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd900009720 tx=0x7fd90002b840 comp rx=0 tx=0).stop 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 -- 172.21.15.138:0/1557631445 shutdown_connections 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 --2- 172.21.15.138:0/1557631445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914117220 0x7fd914119610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 -- 172.21.15.138:0/1557631445 >> 172.21.15.138:0/1557631445 conn(0x7fd914071ae0 msgr2=0x7fd914071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 -- 172.21.15.138:0/1557631445 shutdown_connections 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.014+0000 7fd91b08f640 1 -- 172.21.15.138:0/1557631445 wait complete. 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd91b08f640 1 Processor -- start 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd91b08f640 1 -- start start 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd91b08f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 0x7fd9141a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd91b08f640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141a2ef0 con 0x7fd914114410 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd918e2b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 0x7fd9141a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd918e2b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 0x7fd9141a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36328/0 (socket says 172.21.15.138:36328) 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.015+0000 7fd918e2b640 1 -- 172.21.15.138:0/1130135498 learned_addr learned my addr 172.21.15.138:0/1130135498 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd918e2b640 1 -- 172.21.15.138:0/1130135498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900009400 con 0x7fd914114410 2024-04-27T05:21:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd918e2b640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 0x7fd9141a29b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c420 tx=0x7fd90800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9080074d0 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908007630 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9141a3150 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd908004950 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.016+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9141a7150 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.017+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fd908004b70 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.017+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91404e9b0 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.017+0000 7fd9117fa640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd8f80376b0 0x7fd8f8039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.017+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3845+0+0 (secure 0 0 0) 0x7fd908049790 con 0x7fd914114410 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.017+0000 7fd913fff640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd8f80376b0 0x7fd8f8039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.017+0000 7fd913fff640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd8f80376b0 0x7fd8f8039b70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd900009720 tx=0x7fd900008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:14.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.022+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fd90801ab40 con 0x7fd914114410 2024-04-27T05:21:14.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.196+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) v1 -- 0x7fd914117220 con 0x7fd914114410 2024-04-27T05:21:14.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.890+0000 7fd9117fa640 1 -- 172.21.15.138:0/1130135498 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v18) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd908018030 con 0x7fd914114410 2024-04-27T05:21:14.891 INFO:teuthology.orchestra.run.smithi138.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-27T05:21:14.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.897+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd8f80376b0 msgr2=0x7fd8f8039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:14.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.897+0000 7fd91b08f640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd8f80376b0 0x7fd8f8039b70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd900009720 tx=0x7fd900008030 comp rx=0 tx=0).stop 2024-04-27T05:21:14.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.897+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 msgr2=0x7fd9141a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:14.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.897+0000 7fd91b08f640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 0x7fd9141a29b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd90800c420 tx=0x7fd90800c8e0 comp rx=0 tx=0).stop 2024-04-27T05:21:14.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.898+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 shutdown_connections 2024-04-27T05:21:14.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.898+0000 7fd91b08f640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fd8f80376b0 0x7fd8f8039b70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:14.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.898+0000 7fd91b08f640 1 --2- 172.21.15.138:0/1130135498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd914114410 0x7fd9141a29b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:14.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.898+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 >> 172.21.15.138:0/1130135498 conn(0x7fd914071ae0 msgr2=0x7fd91406be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.901+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 shutdown_connections 2024-04-27T05:21:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:14.904+0000 7fd91b08f640 1 -- 172.21.15.138:0/1130135498 wait complete. 2024-04-27T05:21:14.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-04-27T05:21:15.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3b5f8d640 1 Processor -- start 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3b5f8d640 1 -- start start 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3b5f8d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb3b0115910 0x7fb3b0117d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3b5f8d640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b0118240 con 0x7fb3b0111a40 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3b5f8d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b01193e0 con 0x7fb3b0115910 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3aeffd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb3b0115910 0x7fb3b0117d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3af7fe640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb3b0111a40 0x7fb3b0111e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38294/0 (socket says 172.21.15.138:38294) 2024-04-27T05:21:15.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.018+0000 7fb3af7fe640 1 -- 172.21.15.138:0/2626982310 learned_addr learned my addr 172.21.15.138:0/2626982310 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.019+0000 7fb3aeffd640 1 -- 172.21.15.138:0/2626982310 >> v1:172.21.15.114:6789/0 conn(0x7fb3b0111a40 legacy=0x7fb3b0111e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.019+0000 7fb3aeffd640 1 -- 172.21.15.138:0/2626982310 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0119550 con 0x7fb3b0115910 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.019+0000 7fb3aeffd640 1 --2- 172.21.15.138:0/2626982310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0115910 0x7fb3b0117d00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400a8c0 tx=0x7fb3a402cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce06103ca828d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.019+0000 7fb3ae7fc640 1 -- 172.21.15.138:0/2626982310 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3a402da00 con 0x7fb3b0115910 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.019+0000 7fb3ae7fc640 1 -- 172.21.15.138:0/2626982310 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a402db60 con 0x7fb3b0115910 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3ae7fc640 1 -- 172.21.15.138:0/2626982310 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3a402de20 con 0x7fb3b0115910 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/2626982310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0115910 msgr2=0x7fb3b0117d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 --2- 172.21.15.138:0/2626982310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0115910 0x7fb3b0117d00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400a8c0 tx=0x7fb3a402cd20 comp rx=0 tx=0).stop 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/2626982310 shutdown_connections 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 --2- 172.21.15.138:0/2626982310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0115910 0x7fb3b0117d00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/2626982310 >> 172.21.15.138:0/2626982310 conn(0x7fb3b0071b40 msgr2=0x7fb3b006be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/2626982310 shutdown_connections 2024-04-27T05:21:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.020+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/2626982310 wait complete. 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3b5f8d640 1 Processor -- start 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3b5f8d640 1 -- start start 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3b5f8d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 0x7fb3b01a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3b5f8d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01a0f10 con 0x7fb3b0111a40 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 0x7fb3b01a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3af7fe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 0x7fb3b01a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36342/0 (socket says 172.21.15.138:36342) 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.021+0000 7fb3af7fe640 1 -- 172.21.15.138:0/3795034872 learned_addr learned my addr 172.21.15.138:0/3795034872 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb3af7fe640 1 -- 172.21.15.138:0/3795034872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a400a5a0 con 0x7fb3b0111a40 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb3af7fe640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 0x7fb3b01a09d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e8c0 tx=0x7fb3a000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3a0005ae0 con 0x7fb3b0111a40 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a0005c40 con 0x7fb3b0111a40 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b01a1170 con 0x7fb3b0111a40 2024-04-27T05:21:15.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3a00016a0 con 0x7fb3b0111a40 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.022+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b01a5170 con 0x7fb3b0111a40 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.023+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fb3a0001910 con 0x7fb3b0111a40 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.023+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b004e9b0 con 0x7fb3b0111a40 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.023+0000 7fb397fff640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fb39c0376b0 0x7fb39c039b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.023+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3856+0+0 (secure 0 0 0) 0x7fb3a004c810 con 0x7fb3b0111a40 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.023+0000 7fb3aeffd640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fb39c0376b0 0x7fb39c039b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:15.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.023+0000 7fb3aeffd640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fb39c0376b0 0x7fb39c039b70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4037000 tx=0x7fb3a4031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.027+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fb3a000f360 con 0x7fb3b0111a40 2024-04-27T05:21:15.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.197+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7fb3b0115910 con 0x7fb3b0111a40 2024-04-27T05:21:15.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.893+0000 7fb397fff640 1 -- 172.21.15.138:0/3795034872 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v19) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb3a000f360 con 0x7fb3b0111a40 2024-04-27T05:21:15.895 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'default.rgw.buckets.index' created 2024-04-27T05:21:15.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fb39c0376b0 msgr2=0x7fb39c039b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:15.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fb39c0376b0 0x7fb39c039b70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4037000 tx=0x7fb3a4031000 comp rx=0 tx=0).stop 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 msgr2=0x7fb3b01a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 0x7fb3b01a09d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e8c0 tx=0x7fb3a000ed80 comp rx=0 tx=0).stop 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 shutdown_connections 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fb39c0376b0 0x7fb39c039b70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 --2- 172.21.15.138:0/3795034872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3b0111a40 0x7fb3b01a09d0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e8c0 tx=0x7fb3a000ed80 comp rx=0 tx=0).stop 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 >> 172.21.15.138:0/3795034872 conn(0x7fb3b0071b40 msgr2=0x7fb3b0114460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:15.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.900+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 shutdown_connections 2024-04-27T05:21:15.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:15.901+0000 7fb3b5f8d640 1 -- 172.21.15.138:0/3795034872 wait complete. 2024-04-27T05:21:15.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfcb700640 1 Processor -- start 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfcb700640 1 -- start start 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfcb700640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdfc4114410 0x7fdfc41147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfcb700640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc4114e80 con 0x7fdfc410cd60 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfcb700640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc4115010 con 0x7fdfc4114410 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfc949c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdfc4114410 0x7fdfc41147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfc8c9b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdfc410cd60 0x7fdfc410d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38300/0 (socket says 172.21.15.138:38300) 2024-04-27T05:21:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfc8c9b640 1 -- 172.21.15.138:0/394171360 learned_addr learned my addr 172.21.15.138:0/394171360 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.018+0000 7fdfbbfff640 1 -- 172.21.15.138:0/394171360 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085871865 0 0) 0x7fdfc4114e80 con 0x7fdfc410cd60 2024-04-27T05:21:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.019+0000 7fdfbbfff640 1 -- 172.21.15.138:0/394171360 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb0003520 con 0x7fdfc410cd60 2024-04-27T05:21:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.019+0000 7fdfc949c640 1 -- 172.21.15.138:0/394171360 >> v1:172.21.15.114:6789/0 conn(0x7fdfc410cd60 legacy=0x7fdfc410d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:16.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.019+0000 7fdfc949c640 1 -- 172.21.15.138:0/394171360 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc410d950 con 0x7fdfc4114410 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.019+0000 7fdfc949c640 1 --2- 172.21.15.138:0/394171360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc4114410 0x7fdfc41147f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009720 tx=0x7fdfc002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e6d88e098ca3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.019+0000 7fdfbbfff640 1 -- 172.21.15.138:0/394171360 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfc002e070 con 0x7fdfc4114410 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfbbfff640 1 -- 172.21.15.138:0/394171360 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc00316d0 con 0x7fdfc4114410 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfbbfff640 1 -- 172.21.15.138:0/394171360 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0031970 con 0x7fdfc4114410 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 -- 172.21.15.138:0/394171360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc4114410 msgr2=0x7fdfc41147f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 --2- 172.21.15.138:0/394171360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc4114410 0x7fdfc41147f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009720 tx=0x7fdfc002b840 comp rx=0 tx=0).stop 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 -- 172.21.15.138:0/394171360 shutdown_connections 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 --2- 172.21.15.138:0/394171360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc4114410 0x7fdfc41147f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 -- 172.21.15.138:0/394171360 >> 172.21.15.138:0/394171360 conn(0x7fdfc4071ae0 msgr2=0x7fdfc4071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 -- 172.21.15.138:0/394171360 shutdown_connections 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.020+0000 7fdfcb700640 1 -- 172.21.15.138:0/394171360 wait complete. 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfcb700640 1 Processor -- start 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfcb700640 1 -- start start 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfcb700640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 0x7fdfc41a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfcb700640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc41a51f0 con 0x7fdfc410cd60 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfc949c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 0x7fdfc41a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfc949c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 0x7fdfc41a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36368/0 (socket says 172.21.15.138:36368) 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.021+0000 7fdfc949c640 1 -- 172.21.15.138:0/4068149906 learned_addr learned my addr 172.21.15.138:0/4068149906 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.022+0000 7fdfc949c640 1 -- 172.21.15.138:0/4068149906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc0009400 con 0x7fdfc410cd60 2024-04-27T05:21:16.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.022+0000 7fdfc949c640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 0x7fdfc41a4c20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdfc002b880 tx=0x7fdfc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.022+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0032830 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.022+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc00329b0 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.022+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0032c90 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.022+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc41a53f0 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.023+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc41a9390 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.023+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7fdfc004d3f0 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.023+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc404e9b0 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.024+0000 7fdfb9ffb640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fdfa8037700 0x7fdfa8039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.024+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4220+0+0 (secure 0 0 0) 0x7fdfc006c070 con 0x7fdfc410cd60 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.024+0000 7fdfc8c9b640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fdfa8037700 0x7fdfa8039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.024+0000 7fdfc8c9b640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fdfa8037700 0x7fdfa8039bc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4007fd0 tx=0x7fdfb40010a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:16.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.028+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7fdfc009f7f0 con 0x7fdfc410cd60 2024-04-27T05:21:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.198+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) v1 -- 0x7fdfc4114410 con 0x7fdfc410cd60 2024-04-27T05:21:16.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.896+0000 7fdfb9ffb640 1 -- 172.21.15.138:0/4068149906 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v20) v1 ==== 187+0+0 (secure 0 0 0) 0x7fdfc00308e0 con 0x7fdfc410cd60 2024-04-27T05:21:16.897 INFO:teuthology.orchestra.run.smithi138.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-27T05:21:16.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.903+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fdfa8037700 msgr2=0x7fdfa8039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:16.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.903+0000 7fdfcb700640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fdfa8037700 0x7fdfa8039bc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4007fd0 tx=0x7fdfb40010a0 comp rx=0 tx=0).stop 2024-04-27T05:21:16.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.903+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 msgr2=0x7fdfc41a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:16.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.903+0000 7fdfcb700640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 0x7fdfc41a4c20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdfc002b880 tx=0x7fdfc00046c0 comp rx=0 tx=0).stop 2024-04-27T05:21:16.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.907+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 shutdown_connections 2024-04-27T05:21:16.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.907+0000 7fdfcb700640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7fdfa8037700 0x7fdfa8039bc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:16.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.908+0000 7fdfcb700640 1 --2- 172.21.15.138:0/4068149906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfc410cd60 0x7fdfc41a4c20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:16.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.908+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 >> 172.21.15.138:0/4068149906 conn(0x7fdfc4071ae0 msgr2=0x7fdfc406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:16.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.913+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 shutdown_connections 2024-04-27T05:21:16.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:16.914+0000 7fdfcb700640 1 -- 172.21.15.138:0/4068149906 wait complete. 2024-04-27T05:21:16.933 DEBUG:tasks.rgw:Pools created 2024-04-27T05:21:16.933 INFO:tasks.rgw:Starting rgw... 2024-04-27T05:21:16.933 INFO:tasks.rgw:rgw client.0 config is {} 2024-04-27T05:21:16.933 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-27T05:21:16.933 INFO:tasks.rgw.client.0:Restarting daemon 2024-04-27T05:21:16.933 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-04-27T05:21:16.936 INFO:tasks.rgw.client.0:Started 2024-04-27T05:21:16.936 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi138.front.sepia.ceph.com:80/ 2024-04-27T05:21:16.936 DEBUG:teuthology.orchestra.run.smithi138:> curl http://smithi138.front.sepia.ceph.com:80/ 2024-04-27T05:21:16.963 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T05:21:16.964 INFO:teuthology.orchestra.run.smithi138.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T05:21:16.964 INFO:teuthology.orchestra.run.smithi138.stderr: Dload Upload Total Spent Left Speed 2024-04-27T05:21:16.964 INFO:teuthology.orchestra.run.smithi138.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T05:21:16.964 INFO:teuthology.orchestra.run.smithi138.stderr:curl: (7) Failed to connect to smithi138.front.sepia.ceph.com port 80: Connection refused 2024-04-27T05:21:17.965 DEBUG:teuthology.orchestra.run.smithi138:> curl http://smithi138.front.sepia.ceph.com:80/ 2024-04-27T05:21:17.987 INFO:teuthology.orchestra.run.smithi138.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T05:21:17.987 INFO:teuthology.orchestra.run.smithi138.stderr: Dload Upload Total Spent Left Speed 2024-04-27T05:21:17.988 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T05:21:17.988 INFO:teuthology.orchestra.run.smithi138.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T05:21:17.989 INFO:teuthology.orchestra.run.smithi138.stderr:curl: (7) Failed to connect to smithi138.front.sepia.ceph.com port 80: Connection refused 2024-04-27T05:21:19.990 DEBUG:teuthology.orchestra.run.smithi138:> curl http://smithi138.front.sepia.ceph.com:80/ 2024-04-27T05:21:20.012 INFO:teuthology.orchestra.run.smithi138.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T05:21:20.012 INFO:teuthology.orchestra.run.smithi138.stderr: Dload Upload Total Spent Left Speed 2024-04-27T05:21:20.013 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T05:21:20.014 INFO:teuthology.orchestra.run.smithi138.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T05:21:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:curl: (7) Failed to connect to smithi138.front.sepia.ceph.com port 80: Connection refused 2024-04-27T05:21:24.015 DEBUG:teuthology.orchestra.run.smithi138:> curl http://smithi138.front.sepia.ceph.com:80/ 2024-04-27T05:21:24.037 INFO:teuthology.orchestra.run.smithi138.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T05:21:24.037 INFO:teuthology.orchestra.run.smithi138.stderr: Dload Upload Total Spent Left Speed 2024-04-27T05:21:24.039 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T05:21:24.039 INFO:teuthology.orchestra.run.smithi138.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T05:21:24.039 INFO:teuthology.orchestra.run.smithi138.stderr:curl: (7) Failed to connect to smithi138.front.sepia.ceph.com port 80: Connection refused 2024-04-27T05:21:32.040 DEBUG:teuthology.orchestra.run.smithi138:> curl http://smithi138.front.sepia.ceph.com:80/ 2024-04-27T05:21:32.062 INFO:teuthology.orchestra.run.smithi138.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T05:21:32.062 INFO:teuthology.orchestra.run.smithi138.stderr: Dload Upload Total Spent Left Speed 2024-04-27T05:21:32.065 INFO:teuthology.orchestra.run.smithi138.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 104k 2024-04-27T05:21:32.065 INFO:teuthology.orchestra.run.smithi138.stdout:anonymous 2024-04-27T05:21:32.066 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-04-27T05:21:32.152 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-04-27T05:21:32.152 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-27T05:21:32.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-27T05:21:32.153 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-27T05:21:32.201 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:32.201 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.199+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.199+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f39a0520880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x560fce205400 0x560fce4e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f39a0520880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560fce4dc600 con 0x560fce205000 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f39a0520880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x560fce244280 con 0x560fce205400 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f399c6c6640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x560fce205400 0x560fce4e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f399cec7640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x560fce205000 0x560fce30c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60630/0 (socket says 172.21.15.138:60630) 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f399cec7640 1 -- 172.21.15.138:0/3334749031 learned_addr learned my addr 172.21.15.138:0/3334749031 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f399c6c6640 1 -- 172.21.15.138:0/3334749031 >> v1:172.21.15.114:6789/0 conn(0x560fce205000 legacy=0x560fce30c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.200+0000 7f399c6c6640 1 -- 172.21.15.138:0/3334749031 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce209380 con 0x560fce205400 2024-04-27T05:21:32.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.202+0000 7f399c6c6640 1 --2- 172.21.15.138:0/3334749031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205400 0x560fce4e6580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x560fce310c00 tx=0x560fce316de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925aaeff35c57460 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.202+0000 7f399bec5640 1 -- 172.21.15.138:0/3334749031 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21dce0 con 0x560fce205400 2024-04-27T05:21:32.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.202+0000 7f399bec5640 1 -- 172.21.15.138:0/3334749031 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fce4ddb00 con 0x560fce205400 2024-04-27T05:21:32.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.202+0000 7f399bec5640 1 -- 172.21.15.138:0/3334749031 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21de40 con 0x560fce205400 2024-04-27T05:21:32.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 -- 172.21.15.138:0/3334749031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205400 msgr2=0x560fce4e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 --2- 172.21.15.138:0/3334749031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205400 0x560fce4e6580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x560fce310c00 tx=0x560fce316de0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 -- 172.21.15.138:0/3334749031 shutdown_connections 2024-04-27T05:21:32.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 --2- 172.21.15.138:0/3334749031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205400 0x560fce4e6580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 -- 172.21.15.138:0/3334749031 >> 172.21.15.138:0/3334749031 conn(0x560fce204c00 msgr2=0x560fce30c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 -- 172.21.15.138:0/3334749031 shutdown_connections 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.203+0000 7f39a0520880 1 -- 172.21.15.138:0/3334749031 wait complete. 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f39a0520880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f39a0520880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fce244780 con 0x560fce205000 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f399cec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f399cec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41814/0 (socket says 172.21.15.138:41814) 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f399cec7640 1 -- 172.21.15.138:0/4206206577 learned_addr learned my addr 172.21.15.138:0/4206206577 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.204+0000 7f399cec7640 1 -- 172.21.15.138:0/4206206577 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce208ea0 con 0x560fce205000 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f399cec7640 1 --2- 172.21.15.138:0/4206206577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c0cc0 tx=0x560fce316cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e704670a0824bab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f399b6c4640 1 -- 172.21.15.138:0/4206206577 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21d4a0 con 0x560fce205000 2024-04-27T05:21:32.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f399b6c4640 1 -- 172.21.15.138:0/4206206577 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fce4dde00 con 0x560fce205000 2024-04-27T05:21:32.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f399b6c4640 1 -- 172.21.15.138:0/4206206577 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21dce0 con 0x560fce205000 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 -- 172.21.15.138:0/4206206577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 msgr2=0x560fce4e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 --2- 172.21.15.138:0/4206206577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c0cc0 tx=0x560fce316cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 -- 172.21.15.138:0/4206206577 shutdown_connections 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 --2- 172.21.15.138:0/4206206577 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 -- 172.21.15.138:0/4206206577 >> 172.21.15.138:0/4206206577 conn(0x560fce204c00 msgr2=0x560fce30d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 -- 172.21.15.138:0/4206206577 shutdown_connections 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.205+0000 7f39a0520880 1 -- 172.21.15.138:0/4206206577 wait complete. 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f39a0520880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f39a0520880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fce2448c0 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f399cec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f399cec7640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41820/0 (socket says 172.21.15.138:41820) 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f399cec7640 1 -- 172.21.15.138:0/1167620331 learned_addr learned my addr 172.21.15.138:0/1167620331 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.206+0000 7f399cec7640 1 -- 172.21.15.138:0/1167620331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce2096c0 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f399cec7640 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c0f00 tx=0x560fce316c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f3999ec1640 1 -- 172.21.15.138:0/1167620331 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21de40 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f3999ec1640 1 -- 172.21.15.138:0/1167620331 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fcf0d6000 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560fce2091e0 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f3999ec1640 1 -- 172.21.15.138:0/1167620331 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21d4a0 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fce276ea0 con 0x560fce205000 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.207+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f39a0520880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f39a0520880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fce244c80 con 0x560fceffd800 2024-04-27T05:21:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f3999ec1640 1 -- 172.21.15.138:0/1167620331 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560fce22f600 con 0x560fce205000 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f3999ec1640 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcf164000 0x560fce4e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399d6c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399d6c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41836/0 (socket says 172.21.15.138:41836) 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399d6c8640 1 -- 172.21.15.138:0/204156553 learned_addr learned my addr 172.21.15.138:0/204156553 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f3999ec1640 1 -- 172.21.15.138:0/1167620331 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x560fce217a40 con 0x560fce205000 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399cec7640 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcf164000 0x560fce4e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399d6c8640 1 -- 172.21.15.138:0/204156553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce276d00 con 0x560fceffd800 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399d6c8640 1 --2- 172.21.15.138:0/204156553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c1d40 tx=0x560fce316f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7ad8e48e8373a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f399cec7640 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcf164000 0x560fce4e7080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x560fcf10e480 tx=0x560fce3174a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f39976bc640 1 -- 172.21.15.138:0/204156553 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21de40 con 0x560fceffd800 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f39976bc640 1 -- 172.21.15.138:0/204156553 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fcf0d6300 con 0x560fceffd800 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.208+0000 7f39976bc640 1 -- 172.21.15.138:0/204156553 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21d4a0 con 0x560fceffd800 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 -- 172.21.15.138:0/204156553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 msgr2=0x560fce4e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 --2- 172.21.15.138:0/204156553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c1d40 tx=0x560fce316f00 comp rx=0 tx=0).stop 2024-04-27T05:21:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 -- 172.21.15.138:0/204156553 shutdown_connections 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 --2- 172.21.15.138:0/204156553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 -- 172.21.15.138:0/204156553 >> 172.21.15.138:0/204156553 conn(0x560fce205400 msgr2=0x560fce30f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 -- 172.21.15.138:0/204156553 shutdown_connections 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 -- 172.21.15.138:0/204156553 wait complete. 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.209+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f39a0520880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f39a0520880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fce228280 con 0x560fceffd800 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f399d6c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f399d6c8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41842/0 (socket says 172.21.15.138:41842) 2024-04-27T05:21:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 learned_addr learned my addr 172.21.15.138:0/445268881 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce276b60 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f399d6c8640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c19e0 tx=0x560fce317aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f3995eb9640 1 -- 172.21.15.138:0/445268881 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21de40 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f3995eb9640 1 -- 172.21.15.138:0/445268881 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fcf0d6480 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560fce2769c0 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f3995eb9640 1 -- 172.21.15.138:0/445268881 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fce21d4a0 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.210+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fce276820 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.211+0000 7f3995eb9640 1 -- 172.21.15.138:0/445268881 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560fce22f8c0 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.211+0000 7f3995eb9640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fce58a000 0x560fce4e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.211+0000 7f3995eb9640 1 -- 172.21.15.138:0/445268881 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x560fce217c00 con 0x560fceffd800 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.211+0000 7f399c6c6640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fce58a000 0x560fce4e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.211+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560fceffdc00 0x560fce4e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58a400 con 0x560fceffdc00 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f399d6c8640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560fceffdc00 0x560fce4e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f399c6c6640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fce58a000 0x560fce4e7600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x560fce310c00 tx=0x560fce550120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f399d6c8640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560fceffdc00 0x560fce4e7b80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f39a0520880 20 realm 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58a800 con 0x560fceffdc00 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.212+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58ac00 con 0x560fceffdc00 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58b000 con 0x560fceffdc00 2024-04-27T05:21:32.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x560fce58b400 0x560fce4e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58a400 con 0x560fce58b400 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f399cec7640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x560fce58b400 0x560fce4e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.213+0000 7f399cec7640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x560fce58b400 0x560fce4e8100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f39a0520880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f39a0520880 20 searching for the correct realm 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58a800 con 0x560fce58b400 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f399cec7640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560fce58a800 0x560fce4e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58a400 con 0x560fce58a800 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.214+0000 7f399c6c6640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560fce58a800 0x560fce4e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.215+0000 7f399c6c6640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560fce58a800 0x560fce4e8c00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.215+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.215+0000 7f399c6c6640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x560fce58a400 0x560fce4e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.215+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.215+0000 7f399d6c8640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x560fce58a400 0x560fce4e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x560fce58a400 0x560fce4e9180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.216+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.217+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.217+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.217+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.217+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a800 2024-04-27T05:21:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.217+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.217+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a800 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.218+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.219+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.220+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a800 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.221+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a800 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.222+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58b400 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a400 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fceffdc00 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fceffc800 con 0x560fce58a800 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f39a0520880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f39a0520880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f39a0520880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f39a0520880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.223+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58ac00 con 0x560fceffdc00 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58b000 con 0x560fceffdc00 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58b800 con 0x560fce58b400 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 zone default found 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 4 Realm: () 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 zonegroup default 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.224+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58bc00 con 0x560fce58a400 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8000 con 0x560fce58b400 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8400 con 0x560fce58b400 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 started sync module instance, tier type = 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.225+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8800 con 0x560fce58a800 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.227+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.227+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94625885710592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8c00 con 0x560fce58a800 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.227+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe9000 con 0x560fceffdc00 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.228+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94625885710592] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.228+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.228+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94625885711744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe9400 con 0x560fceffdc00 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.228+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe9800 con 0x560fce58b400 2024-04-27T05:21:32.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.230+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94625885711744] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.232+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.232+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94625885712896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8000 con 0x560fce58b400 2024-04-27T05:21:32.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.232+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8400 con 0x560fce58a800 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.234+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.234+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94625885714048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe9800 con 0x560fce58a800 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.234+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58b800 con 0x560fceffdc00 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.235+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94625885714048] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.235+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94625885712896] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.238+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.238+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94625885715200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58ac00 con 0x560fceffdc00 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.238+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58b000 con 0x560fce58a400 2024-04-27T05:21:32.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.239+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.239+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94625885716352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce58bc00 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.239+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcf165800 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.239+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94625885715200] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.240+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94625885716352] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.240+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.240+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94625885717504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8800 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.240+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8c00 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.241+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94625885717504] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94625885718656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe8400 con 0x560fce58a400 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=0 2024-04-27T05:21:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=1 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=2 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=3 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=4 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=5 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.242+0000 7f39a0520880 20 add_watcher() i=6 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.243+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94625885718656] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.243+0000 7f39a0520880 20 add_watcher() i=7 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.243+0000 7f39a0520880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.243+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe9800 con 0x560fceffdc00 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.243+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.243+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94625885719808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fcefe9000 con 0x560fceffdc00 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.244+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94625885719808] v29'30 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.244+0000 7f3983e95640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.244+0000 7f39a0520880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.244+0000 7f39a0520880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f39a0520880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f39a0520880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fce2288c0 con 0x560fcefe9c00 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f399c6c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f399c6c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41858/0 (socket says 172.21.15.138:41858) 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f399c6c6640 1 -- 172.21.15.138:0/3091033209 learned_addr learned my addr 172.21.15.138:0/3091033209 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.245+0000 7f399c6c6640 1 -- 172.21.15.138:0/3091033209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce276680 con 0x560fcefe9c00 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f399c6c6640 1 --2- 172.21.15.138:0/3091033209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x560fcf17a780 tx=0x560fce5500c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb879f551e31d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f3983694640 1 -- 172.21.15.138:0/3091033209 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fcefea580 con 0x560fcefe9c00 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f3983694640 1 -- 172.21.15.138:0/3091033209 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fcf0d6000 con 0x560fcefe9c00 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f3983694640 1 -- 172.21.15.138:0/3091033209 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fcefea6e0 con 0x560fcefe9c00 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 -- 172.21.15.138:0/3091033209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 msgr2=0x560fce4e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 --2- 172.21.15.138:0/3091033209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x560fcf17a780 tx=0x560fce5500c0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 -- 172.21.15.138:0/3091033209 shutdown_connections 2024-04-27T05:21:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 --2- 172.21.15.138:0/3091033209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 -- 172.21.15.138:0/3091033209 >> 172.21.15.138:0/3091033209 conn(0x560fcefe9400 msgr2=0x560fce30f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 -- 172.21.15.138:0/3091033209 shutdown_connections 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.246+0000 7f39a0520880 1 -- 172.21.15.138:0/3091033209 wait complete. 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f39a0520880 1 Processor -- start 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f39a0520880 1 -- start start 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f39a0520880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f39a0520880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fce228780 con 0x560fcefe9c00 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f399c6c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f399c6c6640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41864/0 (socket says 172.21.15.138:41864) 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f399c6c6640 1 -- 172.21.15.138:0/2675902894 learned_addr learned my addr 172.21.15.138:0/2675902894 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.247+0000 7f399c6c6640 1 -- 172.21.15.138:0/2675902894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fce2764e0 con 0x560fcefe9c00 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.248+0000 7f399c6c6640 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x560fce56b740 tx=0x560fce550000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.248+0000 7f3981e91640 1 -- 172.21.15.138:0/2675902894 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fcefea840 con 0x560fcefe9c00 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.248+0000 7f3981e91640 1 -- 172.21.15.138:0/2675902894 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fcf0d6780 con 0x560fcefe9c00 2024-04-27T05:21:32.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.248+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560fce208ea0 con 0x560fcefe9c00 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.248+0000 7f3981e91640 1 -- 172.21.15.138:0/2675902894 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560fcefea580 con 0x560fcefe9c00 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.248+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fce2096c0 con 0x560fcefe9c00 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f3981e91640 1 -- 172.21.15.138:0/2675902894 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560fce22eb00 con 0x560fcefe9c00 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f3981e91640 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcefe8800 0x560fce4e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f3981e91640 1 -- 172.21.15.138:0/2675902894 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x560fce217dc0 con 0x560fcefe9c00 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f39a0520880 5 note: GC not initialized 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f399d6c8640 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcefe8800 0x560fce4e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f397f68c640 20 reqs_thread_entry: start 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f39a0520880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f399d6c8640 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcefe8800 0x560fce4e9700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x560fcf17abd0 tx=0x560fce550960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f39a0520880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f397d688640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:02 2024 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f397ce66640 20 reqs_thread_entry: start 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.249+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.250+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(62 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.250+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.250+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(63 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.250+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.250+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(64 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.250+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a800 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(67 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.251+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(69 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a800 2024-04-27T05:21:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(70 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(71 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.252+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.253+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(73 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.253+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.253+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(74 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.253+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.253+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(75 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.253+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(76 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(77 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(78 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(79 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.254+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a800 2024-04-27T05:21:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(80 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(81 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(82 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(83 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a800 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(84 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.255+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.256+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(85 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.256+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.256+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(86 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.256+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.256+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(87 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.256+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(88 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a800 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(89 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a400 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(90 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fceffdc00 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(91 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.257+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58b400 2024-04-27T05:21:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.258+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(92 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.258+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a000 con 0x560fce58a800 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.258+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(93 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.258+0000 7f3983e95640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.258+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94625885719808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a400 con 0x560fceffdc00 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.260+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94625885719808] v29'31 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.260+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94625885710592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67a800 con 0x560fce58a800 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.261+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94625885710592] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.261+0000 7f39a0520880 20 remove_watcher() i=0 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.261+0000 7f39a0520880 2 removed watcher, disabling cache 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.261+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94625885711744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67ac00 con 0x560fceffdc00 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.262+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94625885711744] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.262+0000 7f39a0520880 20 remove_watcher() i=1 2024-04-27T05:21:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.262+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94625885712896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67b000 con 0x560fce58b400 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.263+0000 7f399cec7640 1 -- 172.21.15.138:0/445268881 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94625885712896] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59efc0 con 0x560fce58b400 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.263+0000 7f39a0520880 20 remove_watcher() i=2 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.263+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94625885714048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67b400 con 0x560fce58a800 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.264+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94625885714048] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59f200 con 0x560fce58a800 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.264+0000 7f39a0520880 20 remove_watcher() i=3 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.264+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94625885715200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67b800 con 0x560fceffdc00 2024-04-27T05:21:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.266+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94625885715200] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fceffdc00 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.266+0000 7f39a0520880 20 remove_watcher() i=4 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.266+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94625885716352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce67bc00 con 0x560fce58a400 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.266+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94625885716352] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.266+0000 7f39a0520880 20 remove_watcher() i=5 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.266+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94625885717504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce590000 con 0x560fce58a400 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.267+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94625885717504] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.267+0000 7f39a0520880 20 remove_watcher() i=6 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.267+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94625885718656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560fce590400 con 0x560fce58a400 2024-04-27T05:21:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.268+0000 7f399d6c8640 1 -- 172.21.15.138:0/445268881 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94625885718656] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560fce59e900 con 0x560fce58a400 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.268+0000 7f39a0520880 20 remove_watcher() i=7 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x560fce58a400 msgr2=0x560fce4e9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x560fce58a400 0x560fce4e9180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560fceffdc00 msgr2=0x560fce4e7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560fceffdc00 0x560fce4e7b80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560fce58a800 msgr2=0x560fce4e8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560fce58a800 0x560fce4e8c00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x560fce58b400 msgr2=0x560fce4e8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x560fce58b400 0x560fce4e8100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fce58a000 msgr2=0x560fce4e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fce58a000 0x560fce4e7600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x560fce310c00 tx=0x560fce550120 comp rx=0 tx=0).stop 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 msgr2=0x560fce4e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f399c6c6640 1 -- 172.21.15.138:0/445268881 reap_dead start 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c19e0 tx=0x560fce317aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 shutdown_connections 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/445268881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fceffd800 0x560fce4e6b00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 >> 172.21.15.138:0/445268881 conn(0x560fce205400 msgr2=0x560fce30f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 shutdown_connections 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/445268881 wait complete. 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcefe8800 msgr2=0x560fce4e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcefe8800 0x560fce4e9700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x560fcf17abd0 tx=0x560fce550960 comp rx=0 tx=0).stop 2024-04-27T05:21:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 msgr2=0x560fce4e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x560fce56b740 tx=0x560fce550000 comp rx=0 tx=0).stop 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 shutdown_connections 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcefe8800 0x560fce4e9700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 --2- 172.21.15.138:0/2675902894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fcefe9c00 0x560fce4e8680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 >> 172.21.15.138:0/2675902894 conn(0x560fcefe9400 msgr2=0x560fce30f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 shutdown_connections 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.269+0000 7f39a0520880 1 -- 172.21.15.138:0/2675902894 wait complete. 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcf164000 msgr2=0x560fce4e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcf164000 0x560fce4e7080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x560fcf10e480 tx=0x560fce3174a0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 msgr2=0x560fce4e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x560fcf0c0f00 tx=0x560fce316c60 comp rx=0 tx=0).stop 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 shutdown_connections 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560fcf164000 0x560fce4e7080 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 --2- 172.21.15.138:0/1167620331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fce205000 0x560fce4e6580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 >> 172.21.15.138:0/1167620331 conn(0x560fce204c00 msgr2=0x560fce30d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 shutdown_connections 2024-04-27T05:21:32.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.270+0000 7f39a0520880 1 -- 172.21.15.138:0/1167620331 wait complete. 2024-04-27T05:21:32.288 INFO:teuthology.orchestra.run.smithi138.stdout:[] 2024-04-27T05:21:32.293 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T05:21:32.293 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-04-27T05:21:32.293 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-04-27T05:21:32.294 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-04-27T05:21:32.336 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f6f42880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56145191b000 0x561451bfc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f6f42880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561451bf2600 con 0x56145191b400 2024-04-27T05:21:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f6f42880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x56145195a280 con 0x56145191b000 2024-04-27T05:21:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f3948640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56145191b000 0x561451bfc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f3948640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56145191b000 0x561451bfc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41868/0 (socket says 172.21.15.138:41868) 2024-04-27T05:21:32.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f3147640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x56145191b400 0x561451a22800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60644/0 (socket says 172.21.15.138:60644) 2024-04-27T05:21:32.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f3948640 1 -- 172.21.15.138:0/3569064503 learned_addr learned my addr 172.21.15.138:0/3569064503 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f3948640 1 -- 172.21.15.138:0/3569064503 >> v1:172.21.15.114:6789/0 conn(0x56145191b400 legacy=0x561451a22800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.335+0000 7f64f3948640 1 -- 172.21.15.138:0/3569064503 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145191f380 con 0x56145191b000 2024-04-27T05:21:32.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.337+0000 7f64f3948640 1 --2- 172.21.15.138:0/3569064503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x561451a26d50 tx=0x561451a2cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7311f0b1739b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.337+0000 7f64f2946640 1 -- 172.21.15.138:0/3569064503 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451933ce0 con 0x56145191b000 2024-04-27T05:21:32.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.337+0000 7f64f2946640 1 -- 172.21.15.138:0/3569064503 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561451bf3b00 con 0x56145191b000 2024-04-27T05:21:32.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.337+0000 7f64f2946640 1 -- 172.21.15.138:0/3569064503 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451933e40 con 0x56145191b000 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 -- 172.21.15.138:0/3569064503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 msgr2=0x561451bfc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 --2- 172.21.15.138:0/3569064503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x561451a26d50 tx=0x561451a2cde0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 -- 172.21.15.138:0/3569064503 shutdown_connections 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 --2- 172.21.15.138:0/3569064503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 -- 172.21.15.138:0/3569064503 >> 172.21.15.138:0/3569064503 conn(0x56145191ac00 msgr2=0x561451a22000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 -- 172.21.15.138:0/3569064503 shutdown_connections 2024-04-27T05:21:32.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 -- 172.21.15.138:0/3569064503 wait complete. 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.338+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f6f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f6f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145195a780 con 0x56145191b000 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f3948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f3948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41882/0 (socket says 172.21.15.138:41882) 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f3948640 1 -- 172.21.15.138:0/896871173 learned_addr learned my addr 172.21.15.138:0/896871173 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f3948640 1 -- 172.21.15.138:0/896871173 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145191eea0 con 0x56145191b000 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f3948640 1 --2- 172.21.15.138:0/896871173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x56145271f560 tx=0x561451a2ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a970724704fe1884 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f2145640 1 -- 172.21.15.138:0/896871173 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5614519334a0 con 0x56145191b000 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f2145640 1 -- 172.21.15.138:0/896871173 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561451bf3e00 con 0x56145191b000 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.339+0000 7f64f2145640 1 -- 172.21.15.138:0/896871173 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451933ce0 con 0x56145191b000 2024-04-27T05:21:32.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- 172.21.15.138:0/896871173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 msgr2=0x561451bfc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 --2- 172.21.15.138:0/896871173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x56145271f560 tx=0x561451a2ccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- 172.21.15.138:0/896871173 shutdown_connections 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 --2- 172.21.15.138:0/896871173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- 172.21.15.138:0/896871173 >> 172.21.15.138:0/896871173 conn(0x56145191ac00 msgr2=0x561451a23000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- 172.21.15.138:0/896871173 shutdown_connections 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- 172.21.15.138:0/896871173 wait complete. 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f6f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145195a8c0 con 0x56145191b000 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f3948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f3948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41886/0 (socket says 172.21.15.138:41886) 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.340+0000 7f64f3948640 1 -- 172.21.15.138:0/281087874 learned_addr learned my addr 172.21.15.138:0/281087874 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f3948640 1 -- 172.21.15.138:0/281087874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145191f6c0 con 0x56145191b000 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f3948640 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5614527d72c0 tx=0x561451a2cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f0942640 1 -- 172.21.15.138:0/281087874 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451933e40 con 0x56145191b000 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f0942640 1 -- 172.21.15.138:0/281087874 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614526fa000 con 0x56145191b000 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56145191f1e0 con 0x56145191b000 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f0942640 1 -- 172.21.15.138:0/281087874 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5614519334a0 con 0x56145191b000 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.341+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56145198cea0 con 0x56145191b000 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f6f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f6f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145193e280 con 0x561452712c00 2024-04-27T05:21:32.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f0942640 1 -- 172.21.15.138:0/281087874 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561451944b00 con 0x56145191b000 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f4149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f4149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41888/0 (socket says 172.21.15.138:41888) 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f4149640 1 -- 172.21.15.138:0/1810933407 learned_addr learned my addr 172.21.15.138:0/1810933407 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f0942640 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452713c00 0x561451bfd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f0942640 1 -- 172.21.15.138:0/281087874 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56145192da40 con 0x56145191b000 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f3948640 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452713c00 0x561451bfd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f4149640 1 -- 172.21.15.138:0/1810933407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145198cd00 con 0x561452712c00 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.342+0000 7f64f4149640 1 --2- 172.21.15.138:0/1810933407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x561452811800 tx=0x561451a2d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35b721183d1a8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f3948640 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452713c00 0x561451bfd080 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x561452810600 tx=0x561451a2ccc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64ee13d640 1 -- 172.21.15.138:0/1810933407 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451933e40 con 0x561452712c00 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64ee13d640 1 -- 172.21.15.138:0/1810933407 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614526fa600 con 0x561452712c00 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64ee13d640 1 -- 172.21.15.138:0/1810933407 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5614519334a0 con 0x561452712c00 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- 172.21.15.138:0/1810933407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 msgr2=0x561451bfcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 --2- 172.21.15.138:0/1810933407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x561452811800 tx=0x561451a2d980 comp rx=0 tx=0).stop 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- 172.21.15.138:0/1810933407 shutdown_connections 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 --2- 172.21.15.138:0/1810933407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- 172.21.15.138:0/1810933407 >> 172.21.15.138:0/1810933407 conn(0x56145191b400 msgr2=0x561451a25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- 172.21.15.138:0/1810933407 shutdown_connections 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- 172.21.15.138:0/1810933407 wait complete. 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.343+0000 7f64f6f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145193f180 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f4149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f4149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41898/0 (socket says 172.21.15.138:41898) 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 learned_addr learned my addr 172.21.15.138:0/4206434278 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145198cb60 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f4149640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5614527d7470 tx=0x561451a2d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64ec93a640 1 -- 172.21.15.138:0/4206434278 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451933e40 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64ec93a640 1 -- 172.21.15.138:0/4206434278 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614526fa780 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56145198c9c0 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64ec93a640 1 -- 172.21.15.138:0/4206434278 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5614519334a0 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.344+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56145198c820 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64ec93a640 1 -- 172.21.15.138:0/4206434278 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561451945600 con 0x561452712c00 2024-04-27T05:21:32.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64ec93a640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561451ca2000 0x561451bfd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64ec93a640 1 -- 172.21.15.138:0/4206434278 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56145192dc00 con 0x561452712c00 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64f3147640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561451ca2000 0x561451bfd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452713000 0x561451bfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713400 con 0x561452713000 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.345+0000 7f64f4149640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452713000 0x561451bfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f3147640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561451ca2000 0x561451bfd600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x561451a27380 tx=0x561451c68120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f4149640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452713000 0x561451bfdb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 realm 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713800 con 0x561452713000 2024-04-27T05:21:32.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3000 con 0x561452713000 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.346+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3400 con 0x561452713000 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451ca3800 0x561451bfe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3c00 con 0x561451ca3800 2024-04-27T05:21:32.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f3948640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451ca3800 0x561451bfe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f3948640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451ca3800 0x561451bfe100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 20 searching for the correct realm 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.347+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3000 con 0x561451ca3800 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3948640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451ca3000 0x561451bfec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3c00 con 0x561451ca3000 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3147640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451ca3000 0x561451bfec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3147640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451ca3000 0x561451bfec00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3147640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451ca3c00 0x561451bff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.348+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.349+0000 7f64f4149640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451ca3c00 0x561451bff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.349+0000 7f64f4149640 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451ca3c00 0x561451bff180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.349+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.349+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.349+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.349+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.350+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.351+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.352+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.353+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3000 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.354+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3000 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3800 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3c00 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.355+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561452713000 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca2c00 con 0x561451ca3000 2024-04-27T05:21:32.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3400 con 0x561452713000 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.356+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713400 con 0x561452713000 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713800 con 0x561451ca3800 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 zone default found 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 4 Realm: () 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 zonegroup default 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca000 con 0x561451ca3c00 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca400 con 0x561451ca3800 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.357+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca800 con 0x561451ca3800 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.358+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.358+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.358+0000 7f64f6f42880 20 started sync module instance, tier type = 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.358+0000 7f64f6f42880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.358+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccac00 con 0x561451ca3000 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.359+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.359+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94645270825216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccb000 con 0x561451ca3000 2024-04-27T05:21:32.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.359+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccb400 con 0x561452713000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.360+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94645270825216] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.360+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.360+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94645270826368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccb800 con 0x561452713000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.360+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccbc00 con 0x561451ca3800 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.361+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94645270826368] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.361+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.361+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94645270827520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca400 con 0x561451ca3800 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.361+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca800 con 0x561451ca3000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.362+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94645270827520] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.362+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.362+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94645270828672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccbc00 con 0x561451ca3000 2024-04-27T05:21:32.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.362+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca000 con 0x561452713000 2024-04-27T05:21:32.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.363+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94645270828672] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.363+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.363+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94645270829824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713800 con 0x561452713000 2024-04-27T05:21:32.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.363+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713400 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.364+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.364+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94645270830976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ca3400 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.364+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452883800 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.365+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94645270829824] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.365+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94645270830976] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.366+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.366+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94645270832128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452883400 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.366+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccac00 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94645270832128] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94645270833280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccb000 con 0x561451ca3c00 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=0 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=1 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=2 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=3 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=4 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=5 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.367+0000 7f64f6f42880 20 add_watcher() i=6 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.368+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94645270833280] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561451ca3c00 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.368+0000 7f64f6f42880 20 add_watcher() i=7 2024-04-27T05:21:32.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.368+0000 7f64f6f42880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.368+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca800 con 0x561452713000 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.368+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.368+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94645270834432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccb400 con 0x561452713000 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.369+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94645270834432] v29'32 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561451cb8900 con 0x561452713000 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64da916640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f6f42880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f6f42880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f6f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f6f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145193e780 con 0x561451ca2c00 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f3147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f3147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41904/0 (socket says 172.21.15.138:41904) 2024-04-27T05:21:32.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f3147640 1 -- 172.21.15.138:0/3979626725 learned_addr learned my addr 172.21.15.138:0/3979626725 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.370+0000 7f64f3147640 1 -- 172.21.15.138:0/3979626725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145198c680 con 0x561451ca2c00 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f3147640 1 --2- 172.21.15.138:0/3979626725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x561451c7aa20 tx=0x561451c68180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807008c374e0a0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64da115640 1 -- 172.21.15.138:0/3979626725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451ccc580 con 0x561451ca2c00 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64da115640 1 -- 172.21.15.138:0/3979626725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614526fa000 con 0x561451ca2c00 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64da115640 1 -- 172.21.15.138:0/3979626725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451ccc6e0 con 0x561451ca2c00 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 -- 172.21.15.138:0/3979626725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 msgr2=0x561451bfe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 --2- 172.21.15.138:0/3979626725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x561451c7aa20 tx=0x561451c68180 comp rx=0 tx=0).stop 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 -- 172.21.15.138:0/3979626725 shutdown_connections 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 --2- 172.21.15.138:0/3979626725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 -- 172.21.15.138:0/3979626725 >> 172.21.15.138:0/3979626725 conn(0x561451ccb800 msgr2=0x561451a25800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 -- 172.21.15.138:0/3979626725 shutdown_connections 2024-04-27T05:21:32.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.371+0000 7f64f6f42880 1 -- 172.21.15.138:0/3979626725 wait complete. 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f6f42880 1 Processor -- start 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f6f42880 1 -- start start 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f6f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f6f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145193e500 con 0x561451ca2c00 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f3147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f3147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41912/0 (socket says 172.21.15.138:41912) 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 learned_addr learned my addr 172.21.15.138:0/28529433 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145198c4e0 con 0x561451ca2c00 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.372+0000 7f64f3147640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x561451c7b770 tx=0x561451c68ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.373+0000 7f64d8912640 1 -- 172.21.15.138:0/28529433 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451ccc840 con 0x561451ca2c00 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.373+0000 7f64d8912640 1 -- 172.21.15.138:0/28529433 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614526faa80 con 0x561451ca2c00 2024-04-27T05:21:32.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.373+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56145191f380 con 0x561451ca2c00 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.373+0000 7f64d8912640 1 -- 172.21.15.138:0/28529433 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561451ccc580 con 0x561451ca2c00 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.373+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56145191eea0 con 0x561451ca2c00 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.373+0000 7f64d8912640 1 -- 172.21.15.138:0/28529433 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561451945b80 con 0x561451ca2c00 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64d8912640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452883800 0x561451bff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64d8912640 1 -- 172.21.15.138:0/28529433 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56145192ddc0 con 0x561451ca2c00 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64f4149640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452883800 0x561451bff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452883400 0x561451d6a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6e000 con 0x561452883400 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64f3948640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452883400 0x561451d6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64f4149640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452883800 0x561451bff700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x561451c7ae10 tx=0x561451c688a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.374+0000 7f64f3948640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452883400 0x561451d6a000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.375+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'41 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561452883400 2024-04-27T05:21:32.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.375+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451d6e400 0x561451d6a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.375+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6e800 con 0x561451d6e400 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.375+0000 7f64f3147640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451d6e400 0x561451d6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.376+0000 7f64f3147640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451d6e400 0x561451d6a580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.377+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.377+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451d6ec00 0x561451d6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.377+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6f000 con 0x561451d6ec00 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.377+0000 7f64f4149640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451d6ec00 0x561451d6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.377+0000 7f64f4149640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451d6ec00 0x561451d6ab00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.378+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.378+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6f400 con 0x561452883400 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.379+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'33 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561452883400 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.379+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451d6f800 0x561451d6b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.379+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6fc00 con 0x561451d6f800 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.379+0000 7f64f3948640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451d6f800 0x561451d6b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.379+0000 7f64f3948640 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451d6f800 0x561451d6b080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.380+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'26 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.380+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96000 con 0x561451d6ec00 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.381+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'25 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.381+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96400 con 0x561452883400 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.382+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561452883400 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.382+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96800 con 0x561451d6e400 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.383+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'19 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.383+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96c00 con 0x561451d6f800 2024-04-27T05:21:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.385+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.385+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97000 con 0x561451d6ec00 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.385+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.386+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97400 con 0x561451d6f800 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.386+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'27 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.386+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97800 con 0x561451d6e400 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.387+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.387+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97c00 con 0x561451d6f800 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.388+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'16 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.388+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96400 con 0x561451d6e400 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.390+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'33 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.390+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96c00 con 0x561452883400 2024-04-27T05:21:32.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.391+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'34 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561452883400 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.391+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97400 con 0x561451d6ec00 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.392+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'20 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.392+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6e000 con 0x561451d6f800 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.393+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'28 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.393+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6f400 con 0x561451d6ec00 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.394+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.394+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6fc00 con 0x561452883400 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.395+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561452883400 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.395+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451cca400 con 0x561452883400 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.395+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'35 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561452883400 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.396+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96000 con 0x561451d6ec00 2024-04-27T05:21:32.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.397+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.397+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97000 con 0x561451d6f800 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.398+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.398+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97400 con 0x561451d6ec00 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.398+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.398+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96800 con 0x561451d6ec00 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.399+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'28 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.399+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d97800 con 0x561451d6ec00 2024-04-27T05:21:32.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.400+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.400+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d96400 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.401+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'26 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.401+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6f000 con 0x561451d6e400 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.402+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.402+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6f400 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.403+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.403+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451d6e800 con 0x561451d6e400 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.404+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.404+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561452713400 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.405+0000 7f64f4149640 1 -- 172.21.15.138:0/28529433 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'22 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451d6ec00 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.405+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccac00 con 0x561451d6e400 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.407+0000 7f64f3147640 1 -- 172.21.15.138:0/28529433 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'34 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451d6e400 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.407+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451ccb000 con 0x561451d6f800 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.408+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451d6f800 2024-04-27T05:21:32.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.408+0000 7f64d610d640 20 reqs_thread_entry: start 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.408+0000 7f64f6f42880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.408+0000 7f64f6f42880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.408+0000 7f64d4109640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:02 2024 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64d38e7640 20 reqs_thread_entry: start 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f6f42880 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f6f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451dea000 con 0x561451ca3000 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f6f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f6f42880 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-04-27T05:21:32.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64f6f42880 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.409+0000 7f64da916640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.410+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94645270834432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451dea400 con 0x561452713000 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.411+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94645270834432] v29'36 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561452713000 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.411+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94645270825216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451dea800 con 0x561451ca3000 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.413+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94645270825216] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.413+0000 7f64f6f42880 20 remove_watcher() i=0 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.413+0000 7f64f6f42880 2 removed watcher, disabling cache 2024-04-27T05:21:32.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.413+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94645270826368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451deac00 con 0x561452713000 2024-04-27T05:21:32.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.414+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94645270826368] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561452713000 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.414+0000 7f64f6f42880 20 remove_watcher() i=1 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.414+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94645270827520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451deb000 con 0x561451ca3800 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.416+0000 7f64f3948640 1 -- 172.21.15.138:0/4206434278 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94645270827520] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8fc0 con 0x561451ca3800 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.416+0000 7f64f6f42880 20 remove_watcher() i=2 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.416+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94645270828672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451deb400 con 0x561451ca3000 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.417+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94645270828672] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb9200 con 0x561451ca3000 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.417+0000 7f64f6f42880 20 remove_watcher() i=3 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.417+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94645270829824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451deb800 con 0x561452713000 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.418+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94645270829824] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561452713000 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.418+0000 7f64f6f42880 20 remove_watcher() i=4 2024-04-27T05:21:32.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.418+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94645270830976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451debc00 con 0x561451ca3c00 2024-04-27T05:21:32.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.419+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94645270830976] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451ca3c00 2024-04-27T05:21:32.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.419+0000 7f64f6f42880 20 remove_watcher() i=5 2024-04-27T05:21:32.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.419+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94645270832128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451deb000 con 0x561451ca3c00 2024-04-27T05:21:32.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.420+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94645270832128] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451ca3c00 2024-04-27T05:21:32.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.420+0000 7f64f6f42880 20 remove_watcher() i=6 2024-04-27T05:21:32.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.420+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94645270833280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561451dea400 con 0x561451ca3c00 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.422+0000 7f64f4149640 1 -- 172.21.15.138:0/4206434278 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94645270833280] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561451cb8b40 con 0x561451ca3c00 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.422+0000 7f64f6f42880 20 remove_watcher() i=7 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451ca3c00 msgr2=0x561451bff180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451ca3c00 0x561451bff180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452713000 msgr2=0x561451bfdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452713000 0x561451bfdb80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451ca3000 msgr2=0x561451bfec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451ca3000 0x561451bfec00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451ca3800 msgr2=0x561451bfe100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451ca3800 0x561451bfe100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561451ca2000 msgr2=0x561451bfd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561451ca2000 0x561451bfd600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x561451a27380 tx=0x561451c68120 comp rx=0 tx=0).stop 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 msgr2=0x561451bfcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/4206434278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561452712c00 0x561451bfcb00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5614527d7470 tx=0x561451a2d9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 reap_dead start 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f3147640 1 -- 172.21.15.138:0/4206434278 reap_dead start 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 shutdown_connections 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 >> 172.21.15.138:0/4206434278 conn(0x56145191b400 msgr2=0x561451a25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 shutdown_connections 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/4206434278 wait complete. 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451d6e400 msgr2=0x561451d6a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x561451d6e400 0x561451d6a580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452883400 msgr2=0x561451d6a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561452883400 0x561451d6a000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451d6f800 msgr2=0x561451d6b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x561451d6f800 0x561451d6b080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451d6ec00 msgr2=0x561451d6ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561451d6ec00 0x561451d6ab00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452883800 msgr2=0x561451bff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452883800 0x561451bff700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x561451c7ae10 tx=0x561451c688a0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 msgr2=0x561451bfe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f3948640 1 -- 172.21.15.138:0/28529433 reap_dead start 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x561451c7b770 tx=0x561451c68ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 shutdown_connections 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 --2- 172.21.15.138:0/28529433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561451ca2c00 0x561451bfe680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 >> 172.21.15.138:0/28529433 conn(0x561451ccb800 msgr2=0x561451a25800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 shutdown_connections 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.423+0000 7f64f6f42880 1 -- 172.21.15.138:0/28529433 wait complete. 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452713c00 msgr2=0x561451bfd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452713c00 0x561451bfd080 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x561452810600 tx=0x561451a2ccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 msgr2=0x561451bfc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x5614527d72c0 tx=0x561451a2cc60 comp rx=0 tx=0).stop 2024-04-27T05:21:32.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 shutdown_connections 2024-04-27T05:21:32.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561452713c00 0x561451bfd080 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 --2- 172.21.15.138:0/281087874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56145191b000 0x561451bfc580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 >> 172.21.15.138:0/281087874 conn(0x56145191ac00 msgr2=0x561451a23000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 shutdown_connections 2024-04-27T05:21:32.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.424+0000 7f64f6f42880 1 -- 172.21.15.138:0/281087874 wait complete. 2024-04-27T05:21:32.431 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T05:21:32.432 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-04-27T05:21:32.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-04-27T05:21:32.432 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-04-27T05:21:32.514 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:32.514 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.512+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.512+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2c3617880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x558f23731400 0x558f23a12580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2c3617880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558f23a08600 con 0x558f23731000 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2c3617880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x558f23770280 con 0x558f23731400 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2bf3e1640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x558f23731400 0x558f23a12580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2bfbe2640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x558f23731000 0x558f23838800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60652/0 (socket says 172.21.15.138:60652) 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/11336105 learned_addr learned my addr 172.21.15.138:0/11336105 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/11336105 >> v1:172.21.15.114:6789/0 conn(0x558f23731000 legacy=0x558f23838800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.513+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/11336105 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f23735380 con 0x558f23731400 2024-04-27T05:21:32.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/11336105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731400 0x558f23a12580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x558f2383cc00 tx=0x558f23842de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d53905bfd0790cc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2bebe0640 1 -- 172.21.15.138:0/11336105 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f23749ce0 con 0x558f23731400 2024-04-27T05:21:32.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2bebe0640 1 -- 172.21.15.138:0/11336105 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f23a09b00 con 0x558f23731400 2024-04-27T05:21:32.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2bebe0640 1 -- 172.21.15.138:0/11336105 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f23749e40 con 0x558f23731400 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2c3617880 1 -- 172.21.15.138:0/11336105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731400 msgr2=0x558f23a12580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2c3617880 1 --2- 172.21.15.138:0/11336105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731400 0x558f23a12580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x558f2383cc00 tx=0x558f23842de0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2c3617880 1 -- 172.21.15.138:0/11336105 shutdown_connections 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2c3617880 1 --2- 172.21.15.138:0/11336105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731400 0x558f23a12580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2c3617880 1 -- 172.21.15.138:0/11336105 >> 172.21.15.138:0/11336105 conn(0x558f23730c00 msgr2=0x558f23838000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.515+0000 7fb2c3617880 1 -- 172.21.15.138:0/11336105 shutdown_connections 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2c3617880 1 -- 172.21.15.138:0/11336105 wait complete. 2024-04-27T05:21:32.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2c3617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2c3617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f23770780 con 0x558f23731000 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2bfbe2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2bfbe2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41922/0 (socket says 172.21.15.138:41922) 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.516+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1487104983 learned_addr learned my addr 172.21.15.138:0/1487104983 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1487104983 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f23734ea0 con 0x558f23731000 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/1487104983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558f245eccc0 tx=0x558f23842cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfa4477db61e526 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2be3df640 1 -- 172.21.15.138:0/1487104983 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f237494a0 con 0x558f23731000 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2be3df640 1 -- 172.21.15.138:0/1487104983 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f23a09e00 con 0x558f23731000 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2be3df640 1 -- 172.21.15.138:0/1487104983 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f23749ce0 con 0x558f23731000 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 -- 172.21.15.138:0/1487104983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 msgr2=0x558f23a12580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1487104983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558f245eccc0 tx=0x558f23842cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 -- 172.21.15.138:0/1487104983 shutdown_connections 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1487104983 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 -- 172.21.15.138:0/1487104983 >> 172.21.15.138:0/1487104983 conn(0x558f23730c00 msgr2=0x558f23839000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 -- 172.21.15.138:0/1487104983 shutdown_connections 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.517+0000 7fb2c3617880 1 -- 172.21.15.138:0/1487104983 wait complete. 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2c3617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2c3617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f237708c0 con 0x558f23731000 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bfbe2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bfbe2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41932/0 (socket says 172.21.15.138:41932) 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2709053178 learned_addr learned my addr 172.21.15.138:0/2709053178 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2709053178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f237356c0 con 0x558f23731000 2024-04-27T05:21:32.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x558f245ecea0 tx=0x558f23842c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bcbdc640 1 -- 172.21.15.138:0/2709053178 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f23749e40 con 0x558f23731000 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.518+0000 7fb2bcbdc640 1 -- 172.21.15.138:0/2709053178 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f24602000 con 0x558f23731000 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f237351e0 con 0x558f23731000 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2bcbdc640 1 -- 172.21.15.138:0/2709053178 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f237494a0 con 0x558f23731000 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f237a2ea0 con 0x558f23731000 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c3617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c3617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f23754280 con 0x558f24529800 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2bcbdc640 1 -- 172.21.15.138:0/2709053178 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558f2375b600 con 0x558f23731000 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.519+0000 7fb2c03e3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c03e3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41948/0 (socket says 172.21.15.138:41948) 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c03e3640 1 -- 172.21.15.138:0/964465187 learned_addr learned my addr 172.21.15.138:0/964465187 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2bcbdc640 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24690000 0x558f23a13080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2bcbdc640 1 -- 172.21.15.138:0/2709053178 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x558f23743a40 con 0x558f23731000 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24690000 0x558f23a13080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c03e3640 1 -- 172.21.15.138:0/964465187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f237a2d00 con 0x558f24529800 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/964465187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x558f246396b0 tx=0x558f23843a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95e5feee062da03 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24690000 0x558f23a13080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x558f246384e0 tx=0x558f238434a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2ba3d7640 1 -- 172.21.15.138:0/964465187 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f23749e40 con 0x558f24529800 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2ba3d7640 1 -- 172.21.15.138:0/964465187 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f24602300 con 0x558f24529800 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2ba3d7640 1 -- 172.21.15.138:0/964465187 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f237494a0 con 0x558f24529800 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 -- 172.21.15.138:0/964465187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 msgr2=0x558f23a12b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 --2- 172.21.15.138:0/964465187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x558f246396b0 tx=0x558f23843a40 comp rx=0 tx=0).stop 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 -- 172.21.15.138:0/964465187 shutdown_connections 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 --2- 172.21.15.138:0/964465187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 -- 172.21.15.138:0/964465187 >> 172.21.15.138:0/964465187 conn(0x558f23731400 msgr2=0x558f2383b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 -- 172.21.15.138:0/964465187 shutdown_connections 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.520+0000 7fb2c3617880 1 -- 172.21.15.138:0/964465187 wait complete. 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c3617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c3617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f23755180 con 0x558f24529800 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c03e3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c03e3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41962/0 (socket says 172.21.15.138:41962) 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 learned_addr learned my addr 172.21.15.138:0/1276389111 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f237a2b60 con 0x558f24529800 2024-04-27T05:21:32.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.521+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x558f245ed9e0 tx=0x558f23843aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2b8bd4640 1 -- 172.21.15.138:0/1276389111 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f23749e40 con 0x558f24529800 2024-04-27T05:21:32.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2b8bd4640 1 -- 172.21.15.138:0/1276389111 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f24602480 con 0x558f24529800 2024-04-27T05:21:32.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f237a29c0 con 0x558f24529800 2024-04-27T05:21:32.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2b8bd4640 1 -- 172.21.15.138:0/1276389111 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f237494a0 con 0x558f24529800 2024-04-27T05:21:32.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f237a2820 con 0x558f24529800 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2b8bd4640 1 -- 172.21.15.138:0/1276389111 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558f2375b8c0 con 0x558f24529800 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2b8bd4640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f23ab6000 0x558f23a13600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.522+0000 7fb2b8bd4640 1 -- 172.21.15.138:0/1276389111 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x558f23743c00 con 0x558f24529800 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f23ab6000 0x558f23a13600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24529c00 0x558f23a13b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6400 con 0x558f24529c00 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24529c00 0x558f23a13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f23ab6000 0x558f23a13600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x558f2383cc00 tx=0x558f23a7a120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24529c00 0x558f23a13b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 20 realm 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6800 con 0x558f24529c00 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.523+0000 7fb2c3617880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6c00 con 0x558f24529c00 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7000 con 0x558f24529c00 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23ab7400 0x558f23a14100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6400 con 0x558f23ab7400 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23ab7400 0x558f23a14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.524+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23ab7400 0x558f23a14100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2c3617880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2c3617880 20 searching for the correct realm 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6800 con 0x558f23ab7400 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23ab6800 0x558f23a14c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6400 con 0x558f23ab6800 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23ab6800 0x558f23a14c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.525+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23ab6800 0x558f23a14c00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.526+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.526+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23ab6400 0x558f23a15180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.526+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.526+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23ab6400 0x558f23a15180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.526+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23ab6400 0x558f23a15180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.527+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6800 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.528+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6800 2024-04-27T05:21:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.529+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.530+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6800 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.531+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6800 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab7400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6400 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.532+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f24529c00 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24528800 con 0x558f23ab6800 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6c00 con 0x558f24529c00 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7000 con 0x558f24529c00 2024-04-27T05:21:32.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.533+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7800 con 0x558f23ab7400 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 zone default found 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 4 Realm: () 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 zonegroup default 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7c00 con 0x558f23ab6400 2024-04-27T05:21:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514000 con 0x558f23ab7400 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514400 con 0x558f23ab7400 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.534+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.535+0000 7fb2c3617880 20 started sync module instance, tier type = 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.535+0000 7fb2c3617880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.535+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514800 con 0x558f23ab6800 2024-04-27T05:21:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.536+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.536+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94073266415872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514c00 con 0x558f23ab6800 2024-04-27T05:21:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.536+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24515000 con 0x558f24529c00 2024-04-27T05:21:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.537+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94073266415872] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.537+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.537+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94073266417024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24515400 con 0x558f24529c00 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.537+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24515800 con 0x558f23ab7400 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.538+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94073266417024] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.538+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.538+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94073266418176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514000 con 0x558f23ab7400 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.538+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514400 con 0x558f23ab6800 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.539+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.539+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94073266419328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24515800 con 0x558f23ab6800 2024-04-27T05:21:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.539+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7800 con 0x558f24529c00 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.539+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94073266418176] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.540+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94073266419328] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.540+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.540+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94073266420480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab6c00 con 0x558f24529c00 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.540+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7000 con 0x558f23ab6400 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.541+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.541+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94073266421632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7c00 con 0x558f23ab6400 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.541+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24691800 con 0x558f23ab6400 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.541+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94073266420480] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.542+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94073266421632] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.542+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.542+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94073266422784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514800 con 0x558f23ab6400 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.542+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514c00 con 0x558f23ab6400 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94073266423936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514400 con 0x558f23ab6400 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=0 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=1 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=2 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=3 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=4 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=5 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94073266422784] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.543+0000 7fb2c3617880 20 add_watcher() i=6 2024-04-27T05:21:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.544+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94073266423936] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.544+0000 7fb2c3617880 20 add_watcher() i=7 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.544+0000 7fb2c3617880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.544+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24515800 con 0x558f24529c00 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.544+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.544+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94073266425088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24515000 con 0x558f24529c00 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94073266425088] v29'37 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2a6bb0640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c3617880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c3617880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c3617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.545+0000 7fb2c3617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f23754780 con 0x558f24515c00 2024-04-27T05:21:32.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2bf3e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2bf3e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41978/0 (socket says 172.21.15.138:41978) 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/3667433300 learned_addr learned my addr 172.21.15.138:0/3667433300 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/3667433300 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f237a2680 con 0x558f24515c00 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/3667433300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558f246a6780 tx=0x558f23a7a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd38af5ff43c602 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2a63af640 1 -- 172.21.15.138:0/3667433300 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f24516580 con 0x558f24515c00 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2a63af640 1 -- 172.21.15.138:0/3667433300 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f24602000 con 0x558f24515c00 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.546+0000 7fb2a63af640 1 -- 172.21.15.138:0/3667433300 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f245166e0 con 0x558f24515c00 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- 172.21.15.138:0/3667433300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 msgr2=0x558f23a14680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 --2- 172.21.15.138:0/3667433300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558f246a6780 tx=0x558f23a7a0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- 172.21.15.138:0/3667433300 shutdown_connections 2024-04-27T05:21:32.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 --2- 172.21.15.138:0/3667433300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- 172.21.15.138:0/3667433300 >> 172.21.15.138:0/3667433300 conn(0x558f24515400 msgr2=0x558f2383b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- 172.21.15.138:0/3667433300 shutdown_connections 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- 172.21.15.138:0/3667433300 wait complete. 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 Processor -- start 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- start start 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2c3617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f23754500 con 0x558f24515c00 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.547+0000 7fb2bf3e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2bf3e1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:41980/0 (socket says 172.21.15.138:41980) 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 learned_addr learned my addr 172.21.15.138:0/2069391295 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f237a24e0 con 0x558f24515c00 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x558f23a97740 tx=0x558f23a7a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2a4bac640 1 -- 172.21.15.138:0/2069391295 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f24516840 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2a4bac640 1 -- 172.21.15.138:0/2069391295 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558f24602780 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f23734ea0 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2a4bac640 1 -- 172.21.15.138:0/2069391295 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558f24516580 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.548+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f237356c0 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2a4bac640 1 -- 172.21.15.138:0/2069391295 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558f2375ab00 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2a4bac640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24514800 0x558f23a15700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2a4bac640 1 -- 172.21.15.138:0/2069391295 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x558f23743dc0 con 0x558f24515c00 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24514800 0x558f23a15700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24528800 0x558f23b70000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74000 con 0x558f24528800 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24528800 0x558f23b70000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24514800 0x558f23a15700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x558f246a6ba0 tx=0x558f23a7a960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.549+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24528800 0x558f23b70000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.550+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'42 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f24528800 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.550+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23b74400 0x558f23b70580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.550+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74800 con 0x558f23b74400 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.551+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23b74400 0x558f23b70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.551+0000 7fb2bf3e1640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23b74400 0x558f23b70580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.552+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.552+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23b74c00 0x558f23b70b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.552+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75000 con 0x558f23b74c00 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.552+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23b74c00 0x558f23b70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.552+0000 7fb2c03e3640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23b74c00 0x558f23b70b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.553+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.553+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75400 con 0x558f24528800 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.554+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'38 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f24528800 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.554+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23b75800 0x558f23b71080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.554+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75c00 con 0x558f23b75800 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.554+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23b75800 0x558f23b71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.554+0000 7fb2bfbe2640 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23b75800 0x558f23b71080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.555+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'29 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.555+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c000 con 0x558f23b74c00 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.556+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'27 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.556+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c400 con 0x558f24528800 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.557+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f24528800 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.557+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c800 con 0x558f23b74400 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.558+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'21 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.558+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9cc00 con 0x558f23b75800 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.559+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.559+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9d000 con 0x558f23b74c00 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.560+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.560+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9d400 con 0x558f23b75800 2024-04-27T05:21:32.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.561+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'30 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.561+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9d800 con 0x558f23b74400 2024-04-27T05:21:32.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.562+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.562+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9dc00 con 0x558f23b75800 2024-04-27T05:21:32.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.564+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'17 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.564+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c400 con 0x558f23b74400 2024-04-27T05:21:32.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.567+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'35 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.567+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9cc00 con 0x558f24528800 2024-04-27T05:21:32.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.568+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'39 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f24528800 2024-04-27T05:21:32.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.568+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74000 con 0x558f23b74c00 2024-04-27T05:21:32.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.569+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'23 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.569+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75400 con 0x558f23b75800 2024-04-27T05:21:32.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.570+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'31 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.570+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75c00 con 0x558f23b74c00 2024-04-27T05:21:32.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.571+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.571+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514000 con 0x558f24528800 2024-04-27T05:21:32.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.572+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f24528800 2024-04-27T05:21:32.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.572+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75000 con 0x558f24528800 2024-04-27T05:21:32.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.573+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'40 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f24528800 2024-04-27T05:21:32.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.573+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74000 con 0x558f23b74c00 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.574+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.574+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75c00 con 0x558f23b75800 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.575+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.575+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c000 con 0x558f23b74c00 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.576+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.576+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9d000 con 0x558f23b74c00 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.577+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'29 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.577+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514c00 con 0x558f23b74c00 2024-04-27T05:21:32.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.579+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.579+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514400 con 0x558f23b74c00 2024-04-27T05:21:32.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.580+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'28 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.580+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75400 con 0x558f23b74400 2024-04-27T05:21:32.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.581+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.581+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75000 con 0x558f23b74c00 2024-04-27T05:21:32.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.582+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.582+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75c00 con 0x558f23b74400 2024-04-27T05:21:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.585+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'22 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.585+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f24514000 con 0x558f23b74c00 2024-04-27T05:21:32.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.586+0000 7fb2c03e3640 1 -- 172.21.15.138:0/2069391295 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'25 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23b74c00 2024-04-27T05:21:32.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.586+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9d400 con 0x558f23b74400 2024-04-27T05:21:32.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.587+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/2069391295 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'36 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23b74400 2024-04-27T05:21:32.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.587+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9dc00 con 0x558f23b75800 2024-04-27T05:21:32.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.588+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23b75800 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.588+0000 7fb2a23a7640 20 reqs_thread_entry: start 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.588+0000 7fb2c3617880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.588+0000 7fb2c3617880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.588+0000 7fb2a03a3640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:02 2024 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb29fb81640 20 reqs_thread_entry: start 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c3617880 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c3617880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74800 con 0x558f23ab6400 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c3617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c3617880 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.589+0000 7fb2c3617880 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T05:21:32.592 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T05:21:32.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.590+0000 7fb2a6bb0640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.590+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94073266425088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75400 con 0x558f24529c00 2024-04-27T05:21:32.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.590+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94073266425088] v29'41 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.590+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94073266415872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c800 con 0x558f23ab6800 2024-04-27T05:21:32.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.591+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94073266415872] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.591+0000 7fb2c3617880 20 remove_watcher() i=0 2024-04-27T05:21:32.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.591+0000 7fb2c3617880 2 removed watcher, disabling cache 2024-04-27T05:21:32.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.591+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94073266417024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9d800 con 0x558f24529c00 2024-04-27T05:21:32.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.593+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94073266417024] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.593+0000 7fb2c3617880 20 remove_watcher() i=1 2024-04-27T05:21:32.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.593+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94073266418176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b9c400 con 0x558f23ab7400 2024-04-27T05:21:32.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.594+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/1276389111 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94073266418176] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8d80 con 0x558f23ab7400 2024-04-27T05:21:32.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.594+0000 7fb2c3617880 20 remove_watcher() i=2 2024-04-27T05:21:32.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.594+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94073266419328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7c00 con 0x558f23ab6800 2024-04-27T05:21:32.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.594+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94073266419328] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac8fc0 con 0x558f23ab6800 2024-04-27T05:21:32.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.594+0000 7fb2c3617880 20 remove_watcher() i=3 2024-04-27T05:21:32.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.594+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94073266420480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23ab7000 con 0x558f24529c00 2024-04-27T05:21:32.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.596+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94073266420480] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f24529c00 2024-04-27T05:21:32.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.596+0000 7fb2c3617880 20 remove_watcher() i=4 2024-04-27T05:21:32.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.596+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94073266421632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74000 con 0x558f23ab6400 2024-04-27T05:21:32.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.597+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94073266421632] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.597+0000 7fb2c3617880 20 remove_watcher() i=5 2024-04-27T05:21:32.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.597+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94073266422784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b75000 con 0x558f23ab6400 2024-04-27T05:21:32.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.599+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94073266422784] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.600+0000 7fb2c3617880 20 remove_watcher() i=6 2024-04-27T05:21:32.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.600+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94073266423936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558f23b74800 con 0x558f23ab6400 2024-04-27T05:21:32.608 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T05:21:32.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.600+0000 7fb2c03e3640 1 -- 172.21.15.138:0/1276389111 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94073266423936] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558f23ac86c0 con 0x558f23ab6400 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 20 remove_watcher() i=7 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23ab6400 msgr2=0x558f23a15180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23ab6400 0x558f23a15180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24529c00 msgr2=0x558f23a13b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24529c00 0x558f23a13b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23ab6800 msgr2=0x558f23a14c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23ab6800 0x558f23a14c00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23ab7400 msgr2=0x558f23a14100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23ab7400 0x558f23a14100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f23ab6000 msgr2=0x558f23a13600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f23ab6000 0x558f23a13600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x558f2383cc00 tx=0x558f23a7a120 comp rx=0 tx=0).stop 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 msgr2=0x558f23a12b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/1276389111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24529800 0x558f23a12b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x558f245ed9e0 tx=0x558f23843aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 reap_dead start 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2bf3e1640 1 -- 172.21.15.138:0/1276389111 reap_dead start 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 shutdown_connections 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 >> 172.21.15.138:0/1276389111 conn(0x558f23731400 msgr2=0x558f2383b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 shutdown_connections 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/1276389111 wait complete. 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23b74400 msgr2=0x558f23b70580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558f23b74400 0x558f23b70580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24528800 msgr2=0x558f23b70000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558f24528800 0x558f23b70000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23b75800 msgr2=0x558f23b71080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558f23b75800 0x558f23b71080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23b74c00 msgr2=0x558f23b70b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558f23b74c00 0x558f23b70b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24514800 msgr2=0x558f23a15700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24514800 0x558f23a15700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x558f246a6ba0 tx=0x558f23a7a960 comp rx=0 tx=0).stop 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 msgr2=0x558f23a14680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2bfbe2640 1 -- 172.21.15.138:0/2069391295 reap_dead start 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x558f23a97740 tx=0x558f23a7a000 comp rx=0 tx=0).stop 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 shutdown_connections 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2069391295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f24515c00 0x558f23a14680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 >> 172.21.15.138:0/2069391295 conn(0x558f24515400 msgr2=0x558f2383b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.601+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 shutdown_connections 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2069391295 wait complete. 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24690000 msgr2=0x558f23a13080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24690000 0x558f23a13080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x558f246384e0 tx=0x558f238434a0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 msgr2=0x558f23a12580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x558f245ecea0 tx=0x558f23842c60 comp rx=0 tx=0).stop 2024-04-27T05:21:32.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 shutdown_connections 2024-04-27T05:21:32.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558f24690000 0x558f23a13080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 --2- 172.21.15.138:0/2709053178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558f23731000 0x558f23a12580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 >> 172.21.15.138:0/2709053178 conn(0x558f23730c00 msgr2=0x558f23839000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 shutdown_connections 2024-04-27T05:21:32.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.602+0000 7fb2c3617880 1 -- 172.21.15.138:0/2709053178 wait complete. 2024-04-27T05:21:32.612 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T05:21:32.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T05:21:32.612 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T05:21:32.692 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:32.692 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ede62880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ede62880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b827fec600 con 0x55b827d15400 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ede62880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b827d54280 con 0x55b827d15000 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ec607640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ebe06640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55b827d15400 0x55b827e1c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60668/0 (socket says 172.21.15.138:60668) 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ebe06640 1 -- 172.21.15.138:0/1221576928 learned_addr learned my addr 172.21.15.138:0/1221576928 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ec607640 1 -- 172.21.15.138:0/1221576928 >> v1:172.21.15.114:6789/0 conn(0x55b827d15400 legacy=0x55b827e1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.691+0000 7f09ec607640 1 -- 172.21.15.138:0/1221576928 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d19380 con 0x55b827d15000 2024-04-27T05:21:32.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.693+0000 7f09ec607640 1 --2- 172.21.15.138:0/1221576928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55b827e20d50 tx=0x55b827e26de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202d0a0659269c09 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.693+0000 7f09eb605640 1 -- 172.21.15.138:0/1221576928 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2dce0 con 0x55b827d15000 2024-04-27T05:21:32.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.693+0000 7f09eb605640 1 -- 172.21.15.138:0/1221576928 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b827fedb00 con 0x55b827d15000 2024-04-27T05:21:32.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.693+0000 7f09eb605640 1 -- 172.21.15.138:0/1221576928 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2de40 con 0x55b827d15000 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- 172.21.15.138:0/1221576928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 msgr2=0x55b827ff6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 --2- 172.21.15.138:0/1221576928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55b827e20d50 tx=0x55b827e26de0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- 172.21.15.138:0/1221576928 shutdown_connections 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 --2- 172.21.15.138:0/1221576928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- 172.21.15.138:0/1221576928 >> 172.21.15.138:0/1221576928 conn(0x55b827d14c00 msgr2=0x55b827e1c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- 172.21.15.138:0/1221576928 shutdown_connections 2024-04-27T05:21:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- 172.21.15.138:0/1221576928 wait complete. 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.694+0000 7f09ede62880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b827d54780 con 0x55b827d15000 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ec607640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ec607640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42004/0 (socket says 172.21.15.138:42004) 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ec607640 1 -- 172.21.15.138:0/954749151 learned_addr learned my addr 172.21.15.138:0/954749151 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ec607640 1 -- 172.21.15.138:0/954749151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d18ea0 con 0x55b827d15000 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ec607640 1 --2- 172.21.15.138:0/954749151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55b828b19560 tx=0x55b827e26cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ef5ec0eab0390b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09eae04640 1 -- 172.21.15.138:0/954749151 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2d4a0 con 0x55b827d15000 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09eae04640 1 -- 172.21.15.138:0/954749151 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b827fede00 con 0x55b827d15000 2024-04-27T05:21:32.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09eae04640 1 -- 172.21.15.138:0/954749151 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2dce0 con 0x55b827d15000 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ede62880 1 -- 172.21.15.138:0/954749151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 msgr2=0x55b827ff6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.695+0000 7f09ede62880 1 --2- 172.21.15.138:0/954749151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55b828b19560 tx=0x55b827e26cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 -- 172.21.15.138:0/954749151 shutdown_connections 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 --2- 172.21.15.138:0/954749151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 -- 172.21.15.138:0/954749151 >> 172.21.15.138:0/954749151 conn(0x55b827d14c00 msgr2=0x55b827e1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 -- 172.21.15.138:0/954749151 shutdown_connections 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 -- 172.21.15.138:0/954749151 wait complete. 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ede62880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b827d548c0 con 0x55b827d15000 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ec607640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ec607640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42020/0 (socket says 172.21.15.138:42020) 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ec607640 1 -- 172.21.15.138:0/742124793 learned_addr learned my addr 172.21.15.138:0/742124793 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ec607640 1 -- 172.21.15.138:0/742124793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d196c0 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.696+0000 7f09ec607640 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55b828bd0de0 tx=0x55b827e26c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09e9601640 1 -- 172.21.15.138:0/742124793 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2de40 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09e9601640 1 -- 172.21.15.138:0/742124793 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b828af4000 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b827d191e0 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09e9601640 1 -- 172.21.15.138:0/742124793 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2d4a0 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b827d86ea0 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09e9601640 1 -- 172.21.15.138:0/742124793 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b827d3eb00 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09ede62880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09ede62880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b827d38280 con 0x55b828b0cc00 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09e9601640 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828b0dc00 0x55b827ff7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.697+0000 7f09e9601640 1 -- 172.21.15.138:0/742124793 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b827d27a40 con 0x55b827d15000 2024-04-27T05:21:32.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ece08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ec607640 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828b0dc00 0x55b827ff7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ece08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42036/0 (socket says 172.21.15.138:42036) 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ece08640 1 -- 172.21.15.138:0/3892264070 learned_addr learned my addr 172.21.15.138:0/3892264070 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ece08640 1 -- 172.21.15.138:0/3892264070 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d86d00 con 0x55b828b0cc00 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ec607640 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828b0dc00 0x55b827ff7080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55b828c142a0 tx=0x55b827e26cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ece08640 1 --2- 172.21.15.138:0/3892264070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55b828bd15f0 tx=0x55b827e27c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b861edaf68f49c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09e6dfc640 1 -- 172.21.15.138:0/3892264070 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2de40 con 0x55b828b0cc00 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09e6dfc640 1 -- 172.21.15.138:0/3892264070 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b828af4600 con 0x55b828b0cc00 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09e6dfc640 1 -- 172.21.15.138:0/3892264070 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2d4a0 con 0x55b828b0cc00 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.698+0000 7f09ede62880 1 -- 172.21.15.138:0/3892264070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 msgr2=0x55b827ff6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 --2- 172.21.15.138:0/3892264070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55b828bd15f0 tx=0x55b827e27c20 comp rx=0 tx=0).stop 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 -- 172.21.15.138:0/3892264070 shutdown_connections 2024-04-27T05:21:32.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 --2- 172.21.15.138:0/3892264070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 -- 172.21.15.138:0/3892264070 >> 172.21.15.138:0/3892264070 conn(0x55b827d15400 msgr2=0x55b827e1f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 -- 172.21.15.138:0/3892264070 shutdown_connections 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 -- 172.21.15.138:0/3892264070 wait complete. 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ede62880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b827d39180 con 0x55b828b0cc00 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ece08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ece08640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42042/0 (socket says 172.21.15.138:42042) 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 learned_addr learned my addr 172.21.15.138:0/2987716933 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d86b60 con 0x55b828b0cc00 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.699+0000 7f09ece08640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55b828bd1980 tx=0x55b827e27980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.700+0000 7f09e55f9640 1 -- 172.21.15.138:0/2987716933 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2de40 con 0x55b828b0cc00 2024-04-27T05:21:32.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.700+0000 7f09e55f9640 1 -- 172.21.15.138:0/2987716933 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b828af4780 con 0x55b828b0cc00 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.700+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b827d869c0 con 0x55b828b0cc00 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.700+0000 7f09e55f9640 1 -- 172.21.15.138:0/2987716933 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b827d2d4a0 con 0x55b828b0cc00 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.700+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b827d86820 con 0x55b828b0cc00 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.700+0000 7f09e55f9640 1 -- 172.21.15.138:0/2987716933 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b827d3f600 con 0x55b828b0cc00 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09e55f9640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828094000 0x55b827ff7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09e55f9640 1 -- 172.21.15.138:0/2987716933 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b827d27c00 con 0x55b828b0cc00 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ebe06640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828094000 0x55b827ff7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828b0d000 0x55b827ff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d400 con 0x55b828b0d000 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ece08640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828b0d000 0x55b827ff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ebe06640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828094000 0x55b827ff7600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55b828c71260 tx=0x55b82805a120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ece08640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828b0d000 0x55b827ff7b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 20 realm 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.701+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d800 con 0x55b828b0d000 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095000 con 0x55b828b0d000 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095400 con 0x55b828b0d000 2024-04-27T05:21:32.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b828095800 0x55b827ff8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095c00 con 0x55b828095800 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ec607640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b828095800 0x55b827ff8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.702+0000 7f09ec607640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b828095800 0x55b827ff8100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ede62880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ede62880 20 searching for the correct realm 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095000 con 0x55b828095800 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ec607640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b828095000 0x55b827ff8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095c00 con 0x55b828095000 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ebe06640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b828095000 0x55b827ff8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.703+0000 7f09ebe06640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b828095000 0x55b827ff8c00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.704+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.704+0000 7f09ebe06640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b828095c00 0x55b827ff9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.704+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.704+0000 7f09ece08640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b828095c00 0x55b827ff9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.704+0000 7f09ece08640 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b828095c00 0x55b827ff9180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.705+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095000 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.706+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095000 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.707+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.708+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095000 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.709+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095000 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095800 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.710+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095c00 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828b0d000 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828094c00 con 0x55b828095000 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095400 con 0x55b828b0d000 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.711+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d400 con 0x55b828b0d000 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d800 con 0x55b828095800 2024-04-27T05:21:32.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 zone default found 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 4 Realm: () 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 zonegroup default 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8000 con 0x55b828095c00 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8400 con 0x55b828095800 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.712+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8800 con 0x55b828095800 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.713+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.713+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.713+0000 7f09ede62880 20 started sync module instance, tier type = 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.713+0000 7f09ede62880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.713+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8c00 con 0x55b828095000 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.714+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.714+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94249433360640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9000 con 0x55b828095000 2024-04-27T05:21:32.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.714+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9400 con 0x55b828b0d000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.715+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94249433360640] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.715+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.715+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94249433361792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9800 con 0x55b828b0d000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.715+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9c00 con 0x55b828095800 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.716+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94249433361792] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.716+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.716+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94249433362944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8400 con 0x55b828095800 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.716+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8800 con 0x55b828095000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.717+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94249433362944] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.717+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.717+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94249433364096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9c00 con 0x55b828095000 2024-04-27T05:21:32.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.717+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8000 con 0x55b828b0d000 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.719+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.719+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94249433365248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d800 con 0x55b828b0d000 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.719+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d400 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.719+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94249433364096] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.720+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.720+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94249433366400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828095400 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.720+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828c72c00 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.720+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94249433365248] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.721+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94249433366400] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.721+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.721+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94249433367552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828c72800 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.721+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8c00 con 0x55b828095c00 2024-04-27T05:21:32.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94249433367552] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94249433368704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9000 con 0x55b828095c00 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=0 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=1 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=2 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=3 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=4 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=5 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.722+0000 7f09ede62880 20 add_watcher() i=6 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.723+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94249433368704] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.723+0000 7f09ede62880 20 add_watcher() i=7 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.723+0000 7f09ede62880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:32.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.723+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8800 con 0x55b828b0d000 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.723+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.723+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94249433369856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9400 con 0x55b828b0d000 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94249433369856] v29'42 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09d35d5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ede62880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ede62880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ede62880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.724+0000 7f09ede62880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b827d38780 con 0x55b828094c00 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09ebe06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09ebe06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42058/0 (socket says 172.21.15.138:42058) 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09ebe06640 1 -- 172.21.15.138:0/447851261 learned_addr learned my addr 172.21.15.138:0/447851261 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09ebe06640 1 -- 172.21.15.138:0/447851261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d86680 con 0x55b828094c00 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09ebe06640 1 --2- 172.21.15.138:0/447851261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55b828076930 tx=0x55b82805a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a11329075da4450 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09d2dd4640 1 -- 172.21.15.138:0/447851261 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8280ba580 con 0x55b828094c00 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09d2dd4640 1 -- 172.21.15.138:0/447851261 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b828af4000 con 0x55b828094c00 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.725+0000 7f09d2dd4640 1 -- 172.21.15.138:0/447851261 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8280ba6e0 con 0x55b828094c00 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- 172.21.15.138:0/447851261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 msgr2=0x55b827ff8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 --2- 172.21.15.138:0/447851261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55b828076930 tx=0x55b82805a180 comp rx=0 tx=0).stop 2024-04-27T05:21:32.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- 172.21.15.138:0/447851261 shutdown_connections 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 --2- 172.21.15.138:0/447851261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- 172.21.15.138:0/447851261 >> 172.21.15.138:0/447851261 conn(0x55b8280b9800 msgr2=0x55b827e1f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- 172.21.15.138:0/447851261 shutdown_connections 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- 172.21.15.138:0/447851261 wait complete. 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 Processor -- start 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- start start 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ede62880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b827d38500 con 0x55b828094c00 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ebe06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ebe06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42066/0 (socket says 172.21.15.138:42066) 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.726+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 learned_addr learned my addr 172.21.15.138:0/3356321567 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b827d864e0 con 0x55b828094c00 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09ebe06640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55b828077830 tx=0x55b82805aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09d15d1640 1 -- 172.21.15.138:0/3356321567 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8280ba840 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09d15d1640 1 -- 172.21.15.138:0/3356321567 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b828af4a80 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b827d19380 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09d15d1640 1 -- 172.21.15.138:0/3356321567 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b8280ba580 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.727+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b827d18ea0 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09d15d1640 1 -- 172.21.15.138:0/3356321567 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b827d3fb80 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09d15d1640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828c72c00 0x55b827ff9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09d15d1640 1 -- 172.21.15.138:0/3356321567 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b827d27dc0 con 0x55b828094c00 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09ece08640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828c72c00 0x55b827ff9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828c72800 0x55b828158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815c000 con 0x55b828c72800 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09ec607640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828c72800 0x55b828158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09ece08640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828c72c00 0x55b827ff9700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55b828077230 tx=0x55b82805a8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.728+0000 7f09ec607640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828c72800 0x55b828158000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.730+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'43 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828c72800 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.730+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b82815c400 0x55b828158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.730+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815c800 con 0x55b82815c400 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.730+0000 7f09ebe06640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b82815c400 0x55b828158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.730+0000 7f09ebe06640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b82815c400 0x55b828158580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.731+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.731+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b82815cc00 0x55b828158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.731+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815d000 con 0x55b82815cc00 2024-04-27T05:21:32.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.731+0000 7f09ece08640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b82815cc00 0x55b828158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.731+0000 7f09ece08640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b82815cc00 0x55b828158b00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.732+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.732+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815d400 con 0x55b828c72800 2024-04-27T05:21:32.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.733+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'43 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828c72800 2024-04-27T05:21:32.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.733+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b82815d800 0x55b828159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.733+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815dc00 con 0x55b82815d800 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.733+0000 7f09ec607640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b82815d800 0x55b828159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.733+0000 7f09ec607640 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b82815d800 0x55b828159080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.734+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'32 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.734+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174000 con 0x55b82815cc00 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.735+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'29 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.735+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174400 con 0x55b828c72800 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.736+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828c72800 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.736+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174800 con 0x55b82815c400 2024-04-27T05:21:32.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.737+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'23 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.737+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174c00 con 0x55b82815d800 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.738+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.738+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175000 con 0x55b82815cc00 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.739+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.739+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175400 con 0x55b82815d800 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.739+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'33 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.739+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175800 con 0x55b82815c400 2024-04-27T05:21:32.781 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T05:21:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.740+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.740+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175c00 con 0x55b82815d800 2024-04-27T05:21:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.741+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'18 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.741+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174400 con 0x55b82815c400 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.742+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'37 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.742+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174c00 con 0x55b828c72800 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.743+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'44 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828c72800 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.743+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175400 con 0x55b82815cc00 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.744+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'26 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.744+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815c000 con 0x55b82815d800 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.745+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'34 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.745+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815d400 con 0x55b82815cc00 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.746+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.746+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815dc00 con 0x55b828c72800 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.747+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828c72800 2024-04-27T05:21:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.747+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8400 con 0x55b828c72800 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.748+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'45 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828c72800 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.748+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174000 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.749+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.749+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175000 con 0x55b82815d800 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.749+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'28 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.749+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175400 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.750+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.750+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174800 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.751+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'30 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.751+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828175800 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.752+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.752+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828174400 con 0x55b82815cc00 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.753+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'30 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.753+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815d000 con 0x55b82815c400 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.754+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.754+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815d400 con 0x55b82815cc00 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.755+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.755+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b82815c800 con 0x55b82815c400 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.756+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'24 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.756+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b828b0d400 con 0x55b82815cc00 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.757+0000 7f09ece08640 1 -- 172.21.15.138:0/3356321567 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'28 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b82815cc00 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.757+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b8c00 con 0x55b82815c400 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.758+0000 7f09ebe06640 1 -- 172.21.15.138:0/3356321567 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'38 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b82815c400 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.758+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8280b9000 con 0x55b82815d800 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b82815d800 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09cedcc640 20 reqs_thread_entry: start 2024-04-27T05:21:32.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ede62880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ede62880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ccdc8640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:03 2024 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09cc5a6640 20 reqs_thread_entry: start 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ede62880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ede62880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.759+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e2000 con 0x55b828095c00 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.760+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.760+0000 7f09ede62880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.760+0000 7f09ede62880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.760+0000 7f09ede62880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.760+0000 7f09d35d5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.760+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94249433369856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e2400 con 0x55b828b0d000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.761+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94249433369856] v29'46 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.761+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94249433360640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e2800 con 0x55b828095000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.762+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94249433360640] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.762+0000 7f09ede62880 20 remove_watcher() i=0 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.762+0000 7f09ede62880 2 removed watcher, disabling cache 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.762+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94249433361792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e2c00 con 0x55b828b0d000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.763+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94249433361792] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.763+0000 7f09ede62880 20 remove_watcher() i=1 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.763+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94249433362944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e3000 con 0x55b828095800 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.764+0000 7f09ec607640 1 -- 172.21.15.138:0/2987716933 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94249433362944] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aafc0 con 0x55b828095800 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.764+0000 7f09ede62880 20 remove_watcher() i=2 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.764+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94249433364096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e3400 con 0x55b828095000 2024-04-27T05:21:32.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.765+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94249433364096] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280ab200 con 0x55b828095000 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.765+0000 7f09ede62880 20 remove_watcher() i=3 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.765+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94249433365248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e3800 con 0x55b828b0d000 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.766+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94249433365248] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828b0d000 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.766+0000 7f09ede62880 20 remove_watcher() i=4 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.766+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94249433366400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e3c00 con 0x55b828095c00 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.767+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94249433366400] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.767+0000 7f09ede62880 20 remove_watcher() i=5 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.767+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94249433367552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e3000 con 0x55b828095c00 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.768+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94249433367552] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.768+0000 7f09ede62880 20 remove_watcher() i=6 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.768+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94249433368704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b8281e2000 con 0x55b828095c00 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.768+0000 7f09ece08640 1 -- 172.21.15.138:0/2987716933 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94249433368704] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b8280aa900 con 0x55b828095c00 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.768+0000 7f09ede62880 20 remove_watcher() i=7 2024-04-27T05:21:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b828095c00 msgr2=0x55b827ff9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b828095c00 0x55b827ff9180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828b0d000 msgr2=0x55b827ff7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828b0d000 0x55b827ff7b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b828095000 msgr2=0x55b827ff8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b828095000 0x55b827ff8c00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b828095800 msgr2=0x55b827ff8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b828095800 0x55b827ff8100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828094000 msgr2=0x55b827ff7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828094000 0x55b827ff7600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55b828c71260 tx=0x55b82805a120 comp rx=0 tx=0).stop 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 msgr2=0x55b827ff6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/2987716933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828b0cc00 0x55b827ff6b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55b828bd1980 tx=0x55b827e27980 comp rx=0 tx=0).stop 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 reap_dead start 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ebe06640 1 -- 172.21.15.138:0/2987716933 reap_dead start 2024-04-27T05:21:32.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 shutdown_connections 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 >> 172.21.15.138:0/2987716933 conn(0x55b827d15400 msgr2=0x55b827e1f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 shutdown_connections 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/2987716933 wait complete. 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b82815c400 msgr2=0x55b828158580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b82815c400 0x55b828158580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828c72800 msgr2=0x55b828158000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b828c72800 0x55b828158000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b82815d800 msgr2=0x55b828159080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b82815d800 0x55b828159080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b82815cc00 msgr2=0x55b828158b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b82815cc00 0x55b828158b00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828c72c00 msgr2=0x55b827ff9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828c72c00 0x55b827ff9700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55b828077230 tx=0x55b82805a8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ec607640 1 -- 172.21.15.138:0/3356321567 reap_dead start 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 msgr2=0x55b827ff8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55b828077830 tx=0x55b82805aae0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 shutdown_connections 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 --2- 172.21.15.138:0/3356321567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b828094c00 0x55b827ff8680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 >> 172.21.15.138:0/3356321567 conn(0x55b8280b9800 msgr2=0x55b827e1f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 shutdown_connections 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.769+0000 7f09ede62880 1 -- 172.21.15.138:0/3356321567 wait complete. 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828b0dc00 msgr2=0x55b827ff7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828b0dc00 0x55b827ff7080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55b828c142a0 tx=0x55b827e26cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 msgr2=0x55b827ff6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55b828bd0de0 tx=0x55b827e26c60 comp rx=0 tx=0).stop 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 shutdown_connections 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b828b0dc00 0x55b827ff7080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 --2- 172.21.15.138:0/742124793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b827d15000 0x55b827ff6580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 >> 172.21.15.138:0/742124793 conn(0x55b827d14c00 msgr2=0x55b827e1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 shutdown_connections 2024-04-27T05:21:32.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.770+0000 7f09ede62880 1 -- 172.21.15.138:0/742124793 wait complete. 2024-04-27T05:21:32.791 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T05:21:32.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T05:21:32.791 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T05:21:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:32.864 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae793c880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a769577000 0x55a769858580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae793c880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a76984e600 con 0x55a769577400 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae793c880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a7695b6280 con 0x55a769577000 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae433f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a769577000 0x55a769858580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae433f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a769577000 0x55a769858580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42070/0 (socket says 172.21.15.138:42070) 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae433f640 1 -- 172.21.15.138:0/2206616268 learned_addr learned my addr 172.21.15.138:0/2206616268 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae3b3e640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a769577400 0x55a76967e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60672/0 (socket says 172.21.15.138:60672) 2024-04-27T05:21:32.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae433f640 1 -- 172.21.15.138:0/2206616268 >> v1:172.21.15.114:6789/0 conn(0x55a769577400 legacy=0x55a76967e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.862+0000 7fcae433f640 1 -- 172.21.15.138:0/2206616268 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76957b380 con 0x55a769577000 2024-04-27T05:21:32.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.864+0000 7fcae433f640 1 --2- 172.21.15.138:0/2206616268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55a769682d50 tx=0x55a769688de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31795223621a9ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.864+0000 7fcae333d640 1 -- 172.21.15.138:0/2206616268 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958fce0 con 0x55a769577000 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.864+0000 7fcae333d640 1 -- 172.21.15.138:0/2206616268 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76984fb00 con 0x55a769577000 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.864+0000 7fcae333d640 1 -- 172.21.15.138:0/2206616268 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958fe40 con 0x55a769577000 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 -- 172.21.15.138:0/2206616268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 msgr2=0x55a769858580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 --2- 172.21.15.138:0/2206616268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55a769682d50 tx=0x55a769688de0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 -- 172.21.15.138:0/2206616268 shutdown_connections 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 --2- 172.21.15.138:0/2206616268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 -- 172.21.15.138:0/2206616268 >> 172.21.15.138:0/2206616268 conn(0x55a769576c00 msgr2=0x55a76967e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 -- 172.21.15.138:0/2206616268 shutdown_connections 2024-04-27T05:21:32.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.865+0000 7fcae793c880 1 -- 172.21.15.138:0/2206616268 wait complete. 2024-04-27T05:21:32.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae793c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae793c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7695b6780 con 0x55a769577000 2024-04-27T05:21:32.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae433f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae433f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42086/0 (socket says 172.21.15.138:42086) 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae433f640 1 -- 172.21.15.138:0/1833316530 learned_addr learned my addr 172.21.15.138:0/1833316530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae433f640 1 -- 172.21.15.138:0/1833316530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76957aea0 con 0x55a769577000 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae433f640 1 --2- 172.21.15.138:0/1833316530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55a76a37b560 tx=0x55a769688cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6b71b65242004e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae2b3c640 1 -- 172.21.15.138:0/1833316530 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958f4a0 con 0x55a769577000 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae2b3c640 1 -- 172.21.15.138:0/1833316530 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76984fe00 con 0x55a769577000 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.866+0000 7fcae2b3c640 1 -- 172.21.15.138:0/1833316530 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958fce0 con 0x55a769577000 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- 172.21.15.138:0/1833316530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 msgr2=0x55a769858580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 --2- 172.21.15.138:0/1833316530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55a76a37b560 tx=0x55a769688cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- 172.21.15.138:0/1833316530 shutdown_connections 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 --2- 172.21.15.138:0/1833316530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577000 0x55a769858580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- 172.21.15.138:0/1833316530 >> 172.21.15.138:0/1833316530 conn(0x55a769576c00 msgr2=0x55a76967f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- 172.21.15.138:0/1833316530 shutdown_connections 2024-04-27T05:21:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- 172.21.15.138:0/1833316530 wait complete. 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 0x55a769858b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.867+0000 7fcae793c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7695b68c0 con 0x55a769577400 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae433f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 0x55a769858b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae433f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 0x55a769858b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42102/0 (socket says 172.21.15.138:42102) 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae433f640 1 -- 172.21.15.138:0/2248185699 learned_addr learned my addr 172.21.15.138:0/2248185699 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae433f640 1 -- 172.21.15.138:0/2248185699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a76957b6c0 con 0x55a769577400 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae433f640 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 0x55a769858b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55a76a4334d0 tx=0x55a769688c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae1339640 1 -- 172.21.15.138:0/2248185699 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958fe40 con 0x55a769577400 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae1339640 1 -- 172.21.15.138:0/2248185699 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76a436000 con 0x55a769577400 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a76957b1e0 con 0x55a769577400 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae1339640 1 -- 172.21.15.138:0/2248185699 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958f4a0 con 0x55a769577400 2024-04-27T05:21:32.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.868+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7695e8ea0 con 0x55a769577400 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae1339640 1 -- 172.21.15.138:0/2248185699 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a7695a0b00 con 0x55a769577400 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae793c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae793c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76959a280 con 0x55a76a36f000 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae1339640 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a76a36fc00 0x55a76985a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae4b40640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae1339640 1 -- 172.21.15.138:0/2248185699 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a769589a40 con 0x55a769577400 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae4b40640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42104/0 (socket says 172.21.15.138:42104) 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae4b40640 1 -- 172.21.15.138:0/339138652 learned_addr learned my addr 172.21.15.138:0/339138652 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae433f640 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a76a36fc00 0x55a76985a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae4b40640 1 -- 172.21.15.138:0/339138652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7695e8d00 con 0x55a76a36f000 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae433f640 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a76a36fc00 0x55a76985a100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55a76a4689c0 tx=0x55a769688cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.869+0000 7fcae4b40640 1 --2- 172.21.15.138:0/339138652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55a76a433b60 tx=0x55a769689c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f3748a5506a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcadeb34640 1 -- 172.21.15.138:0/339138652 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958fe40 con 0x55a76a36f000 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcadeb34640 1 -- 172.21.15.138:0/339138652 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76a436600 con 0x55a76a36f000 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcadeb34640 1 -- 172.21.15.138:0/339138652 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958f4a0 con 0x55a76a36f000 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 -- 172.21.15.138:0/339138652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 msgr2=0x55a769859080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 --2- 172.21.15.138:0/339138652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55a76a433b60 tx=0x55a769689c20 comp rx=0 tx=0).stop 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 -- 172.21.15.138:0/339138652 shutdown_connections 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 --2- 172.21.15.138:0/339138652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 -- 172.21.15.138:0/339138652 >> 172.21.15.138:0/339138652 conn(0x55a76a36ec00 msgr2=0x55a769681000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 -- 172.21.15.138:0/339138652 shutdown_connections 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 -- 172.21.15.138:0/339138652 wait complete. 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.870+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae793c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae793c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76959b180 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae4b40640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae4b40640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42108/0 (socket says 172.21.15.138:42108) 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 learned_addr learned my addr 172.21.15.138:0/1792665116 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7695e8b60 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55a76a433a70 tx=0x55a7696891a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcadd331640 1 -- 172.21.15.138:0/1792665116 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958fe40 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcadd331640 1 -- 172.21.15.138:0/1792665116 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76a436780 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7695e89c0 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcadd331640 1 -- 172.21.15.138:0/1792665116 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a76958f4a0 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.871+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7695e8820 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcadd331640 1 -- 172.21.15.138:0/1792665116 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a7695a1600 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcadd331640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a7698f8000 0x55a76985a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcadd331640 1 -- 172.21.15.138:0/1792665116 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a769589c00 con 0x55a76a36f000 2024-04-27T05:21:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a7698f8000 0x55a76985a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a76a36f400 0x55a769859600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a36f800 con 0x55a76a36f400 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.872+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a76a36f400 0x55a769859600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a7698f8000 0x55a76985a680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a769683380 tx=0x55a7698c0120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a76a36f400 0x55a769859600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 realm 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9000 con 0x55a76a36f400 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9400 con 0x55a76a36f400 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9800 con 0x55a76a36f400 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a7698f9c00 0x55a769859b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.873+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9000 con 0x55a7698f9c00 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae433f640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a7698f9c00 0x55a769859b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae433f640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a7698f9c00 0x55a769859b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae793c880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae793c880 20 searching for the correct realm 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9400 con 0x55a7698f9c00 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.874+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae433f640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a7698f9400 0x55a769858580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9000 con 0x55a7698f9400 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a7698f9400 0x55a769858580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a7698f9400 0x55a769858580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a7698f9000 0x55a76985ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.875+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a7698f9000 0x55a76985ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a7698f9000 0x55a76985ac00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.876+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9400 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.877+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9400 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.878+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.879+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9400 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.880+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.881+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9400 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9c00 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9000 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a7698f9400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae793c880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae793c880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae793c880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae793c880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.882+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f9800 con 0x55a76a36f400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a36f800 con 0x55a76a36f400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922000 con 0x55a7698f9c00 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 zone default found 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 4 Realm: () 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 zonegroup default 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.883+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922400 con 0x55a7698f9000 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922800 con 0x55a7698f9c00 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922c00 con 0x55a7698f9c00 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 started sync module instance, tier type = 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.884+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923000 con 0x55a7698f9400 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.885+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.885+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94177518225664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923400 con 0x55a7698f9400 2024-04-27T05:21:32.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.885+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923800 con 0x55a76a36f400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.886+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94177518225664] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.886+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.886+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94177518226816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922000 con 0x55a76a36f400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.886+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922800 con 0x55a7698f9c00 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.887+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94177518226816] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.887+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.887+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94177518227968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922c00 con 0x55a7698f9c00 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.887+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923c00 con 0x55a7698f9400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.888+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.888+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94177518229120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a4d9800 con 0x55a7698f9400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.888+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a4d9400 con 0x55a76a36f400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.889+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94177518227968] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.889+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.889+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94177518230272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a4d9000 con 0x55a76a36f400 2024-04-27T05:21:32.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.889+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923000 con 0x55a7698f9000 2024-04-27T05:21:32.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.890+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94177518229120] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.890+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.890+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94177518231424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923400 con 0x55a7698f9000 2024-04-27T05:21:32.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.890+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923c00 con 0x55a7698f9000 2024-04-27T05:21:32.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.890+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94177518230272] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.891+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94177518231424] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.891+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.891+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94177518232576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922400 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.891+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923800 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94177518232576] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94177518233728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922000 con 0x55a7698f9000 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=0 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=1 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=2 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=3 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=4 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=5 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.892+0000 7fcae793c880 20 add_watcher() i=6 2024-04-27T05:21:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.893+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94177518233728] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a7698f9000 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.893+0000 7fcae793c880 20 add_watcher() i=7 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.893+0000 7fcae793c880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.893+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a36f800 con 0x55a76a36f400 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.893+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.893+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94177518234880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7698f8c00 con 0x55a76a36f400 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.894+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94177518234880] v29'47 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a769910900 con 0x55a76a36f400 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.894+0000 7fcacb30d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae793c880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae793c880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae793c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae793c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76959a780 con 0x55a769923000 2024-04-27T05:21:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae3b3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae3b3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42118/0 (socket says 172.21.15.138:42118) 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae3b3e640 1 -- 172.21.15.138:0/2639035746 learned_addr learned my addr 172.21.15.138:0/2639035746 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.895+0000 7fcae3b3e640 1 -- 172.21.15.138:0/2639035746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7695e8680 con 0x55a769923000 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/2639035746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55a76a4e67b0 tx=0x55a7698c0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f618aec944977151 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcacab0c640 1 -- 172.21.15.138:0/2639035746 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a769924580 con 0x55a769923000 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcacab0c640 1 -- 172.21.15.138:0/2639035746 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76a436000 con 0x55a769923000 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcacab0c640 1 -- 172.21.15.138:0/2639035746 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a7699246e0 con 0x55a769923000 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 -- 172.21.15.138:0/2639035746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 msgr2=0x55a76985b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 --2- 172.21.15.138:0/2639035746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55a76a4e67b0 tx=0x55a7698c0180 comp rx=0 tx=0).stop 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 -- 172.21.15.138:0/2639035746 shutdown_connections 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 --2- 172.21.15.138:0/2639035746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 -- 172.21.15.138:0/2639035746 >> 172.21.15.138:0/2639035746 conn(0x55a7698f9800 msgr2=0x55a769681800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 -- 172.21.15.138:0/2639035746 shutdown_connections 2024-04-27T05:21:32.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.896+0000 7fcae793c880 1 -- 172.21.15.138:0/2639035746 wait complete. 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae793c880 1 Processor -- start 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae793c880 1 -- start start 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae793c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae793c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76959a500 con 0x55a769923000 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae3b3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae3b3e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42122/0 (socket says 172.21.15.138:42122) 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 learned_addr learned my addr 172.21.15.138:0/1876307798 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7695e84e0 con 0x55a769923000 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55a7698dd800 tx=0x55a7698c0ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcac9309640 1 -- 172.21.15.138:0/1876307798 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a769924840 con 0x55a769923000 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcac9309640 1 -- 172.21.15.138:0/1876307798 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a76a436a80 con 0x55a769923000 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.897+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a76957b380 con 0x55a769923000 2024-04-27T05:21:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcac9309640 1 -- 172.21.15.138:0/1876307798 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a769924580 con 0x55a769923000 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a76957aea0 con 0x55a769923000 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcac9309640 1 -- 172.21.15.138:0/1876307798 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a7695a1b80 con 0x55a769923000 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcac9309640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a769931000 0x55a7699ba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcac9309640 1 -- 172.21.15.138:0/1876307798 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a769589dc0 con 0x55a769923000 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a769931000 0x55a7699ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a769923400 0x55a76985b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.898+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923c00 con 0x55a769923400 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.899+0000 7fcae433f640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a769923400 0x55a76985b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.899+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a769931000 0x55a7699ba000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a7698dcf90 tx=0x55a7698c08a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.899+0000 7fcae433f640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a769923400 0x55a76985b700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.900+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'44 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769923400 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.900+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a769922400 0x55a7699ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.900+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a4d9400 con 0x55a769922400 2024-04-27T05:21:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.900+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a769922400 0x55a7699ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.901+0000 7fcae3b3e640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a769922400 0x55a7699ba580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.901+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.901+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a76a4d9000 0x55a7699bab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.901+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769931400 con 0x55a76a4d9000 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.902+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a76a4d9000 0x55a7699bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.902+0000 7fcae4b40640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a76a4d9000 0x55a7699bab00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.903+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.903+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769931800 con 0x55a769923400 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.903+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'48 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769923400 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.903+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a769931c00 0x55a7699bb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.904+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de000 con 0x55a769931c00 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.904+0000 7fcae433f640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a769931c00 0x55a7699bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.904+0000 7fcae433f640 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a769931c00 0x55a7699bb080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.905+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'35 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.905+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de400 con 0x55a76a4d9000 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.905+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'31 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.906+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de800 con 0x55a769923400 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.907+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769923400 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.907+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699dec00 con 0x55a769922400 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.908+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'25 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.908+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df000 con 0x55a769931c00 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.909+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.909+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df400 con 0x55a76a4d9000 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.910+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.910+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df800 con 0x55a769931c00 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.911+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'36 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.911+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699dfc00 con 0x55a769922400 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.912+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.912+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de000 con 0x55a769931c00 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.913+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'19 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.913+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de800 con 0x55a769922400 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.914+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'39 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.914+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df000 con 0x55a769923400 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.915+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'49 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769923400 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.915+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769931800 con 0x55a76a4d9000 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.916+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'29 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.916+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922800 con 0x55a769931c00 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.917+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'37 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.917+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922c00 con 0x55a76a4d9000 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.917+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.917+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923c00 con 0x55a769923400 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.919+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769923400 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.919+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de400 con 0x55a769923400 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.919+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'50 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769923400 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.920+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df400 con 0x55a76a4d9000 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.920+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'30 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.920+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699dec00 con 0x55a769931c00 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.921+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'29 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.921+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699dfc00 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.922+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.922+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de800 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.923+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'31 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.923+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a76a4d9400 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.924+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.924+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769931400 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.925+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'32 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.925+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769931800 con 0x55a769922400 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.926+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.926+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769930c00 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.927+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.927+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769930800 con 0x55a769922400 2024-04-27T05:21:32.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.928+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'26 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.928+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923800 con 0x55a76a4d9000 2024-04-27T05:21:32.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.929+0000 7fcae4b40640 1 -- 172.21.15.138:0/1876307798 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'31 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a4d9000 2024-04-27T05:21:32.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.929+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922000 con 0x55a769922400 2024-04-27T05:21:32.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.930+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1876307798 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'40 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a769922400 2024-04-27T05:21:32.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.930+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922c00 con 0x55a769931c00 2024-04-27T05:21:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.931+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a769931c00 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.931+0000 7fcac6b04640 20 reqs_thread_entry: start 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.931+0000 7fcae793c880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.931+0000 7fcae793c880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.931+0000 7fcac4b00640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:03 2024 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.931+0000 7fcac42de640 20 reqs_thread_entry: start 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923c00 con 0x55a76a36f400 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a36f400 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-27T05:21:32.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:32.936 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T05:21:32.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcacb30d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:32.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.932+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94177518234880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769922c00 con 0x55a76a36f400 2024-04-27T05:21:32.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.933+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94177518234880] v29'51 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a36f400 2024-04-27T05:21:32.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.933+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94177518225664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df800 con 0x55a7698f9400 2024-04-27T05:21:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.934+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94177518225664] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.934+0000 7fcae793c880 20 remove_watcher() i=0 2024-04-27T05:21:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.934+0000 7fcae793c880 2 removed watcher, disabling cache 2024-04-27T05:21:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.934+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94177518226816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de000 con 0x55a76a36f400 2024-04-27T05:21:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.935+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94177518226816] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a36f400 2024-04-27T05:21:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.935+0000 7fcae793c880 20 remove_watcher() i=1 2024-04-27T05:21:32.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.935+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94177518227968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699df000 con 0x55a7698f9c00 2024-04-27T05:21:32.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.936+0000 7fcae433f640 1 -- 172.21.15.138:0/1792665116 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94177518227968] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910fc0 con 0x55a7698f9c00 2024-04-27T05:21:32.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.936+0000 7fcae793c880 20 remove_watcher() i=2 2024-04-27T05:21:32.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.936+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94177518229120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699de400 con 0x55a7698f9400 2024-04-27T05:21:32.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.937+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94177518229120] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769911200 con 0x55a7698f9400 2024-04-27T05:21:32.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.937+0000 7fcae793c880 20 remove_watcher() i=3 2024-04-27T05:21:32.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.937+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94177518230272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a7699dec00 con 0x55a76a36f400 2024-04-27T05:21:32.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.938+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94177518230272] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a76a36f400 2024-04-27T05:21:32.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.938+0000 7fcae793c880 20 remove_watcher() i=4 2024-04-27T05:21:32.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.938+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94177518231424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923800 con 0x55a7698f9000 2024-04-27T05:21:32.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.939+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94177518231424] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a7698f9000 2024-04-27T05:21:32.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.939+0000 7fcae793c880 20 remove_watcher() i=5 2024-04-27T05:21:32.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.939+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94177518232576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769923c00 con 0x55a7698f9000 2024-04-27T05:21:32.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.940+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94177518232576] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a7698f9000 2024-04-27T05:21:32.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.940+0000 7fcae793c880 20 remove_watcher() i=6 2024-04-27T05:21:32.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.940+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94177518233728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a769931400 con 0x55a7698f9000 2024-04-27T05:21:32.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae4b40640 1 -- 172.21.15.138:0/1792665116 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94177518233728] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a769910b40 con 0x55a7698f9000 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 20 remove_watcher() i=7 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a7698f9000 msgr2=0x55a76985ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a7698f9000 0x55a76985ac00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a76a36f400 msgr2=0x55a769859600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a76a36f400 0x55a769859600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a7698f9400 msgr2=0x55a769858580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a7698f9400 0x55a769858580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a7698f9c00 msgr2=0x55a769859b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a7698f9c00 0x55a769859b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a7698f8000 msgr2=0x55a76985a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a7698f8000 0x55a76985a680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a769683380 tx=0x55a7698c0120 comp rx=0 tx=0).stop 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 msgr2=0x55a769859080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1792665116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a76a36f000 0x55a769859080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55a76a433a70 tx=0x55a7696891a0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 reap_dead start 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae3b3e640 1 -- 172.21.15.138:0/1792665116 reap_dead start 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 shutdown_connections 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 >> 172.21.15.138:0/1792665116 conn(0x55a76a36ec00 msgr2=0x55a769681000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 shutdown_connections 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1792665116 wait complete. 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a769922400 msgr2=0x55a7699ba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a769922400 0x55a7699ba580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a769923400 msgr2=0x55a76985b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a769923400 0x55a76985b700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a769931c00 msgr2=0x55a7699bb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a769931c00 0x55a7699bb080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a76a4d9000 msgr2=0x55a7699bab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a76a4d9000 0x55a7699bab00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a769931000 msgr2=0x55a7699ba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.941+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a769931000 0x55a7699ba000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a7698dcf90 tx=0x55a7698c08a0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae433f640 1 -- 172.21.15.138:0/1876307798 reap_dead start 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 msgr2=0x55a76985b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55a7698dd800 tx=0x55a7698c0ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 shutdown_connections 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 --2- 172.21.15.138:0/1876307798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769923000 0x55a76985b180 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 >> 172.21.15.138:0/1876307798 conn(0x55a7698f9800 msgr2=0x55a769681800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 shutdown_connections 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/1876307798 wait complete. 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a76a36fc00 msgr2=0x55a76985a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a76a36fc00 0x55a76985a100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55a76a4689c0 tx=0x55a769688cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 msgr2=0x55a769858b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 0x55a769858b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55a76a4334d0 tx=0x55a769688c60 comp rx=0 tx=0).stop 2024-04-27T05:21:32.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 shutdown_connections 2024-04-27T05:21:32.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a76a36fc00 0x55a76985a100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 --2- 172.21.15.138:0/2248185699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a769577400 0x55a769858b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:32.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 >> 172.21.15.138:0/2248185699 conn(0x55a769576c00 msgr2=0x55a76967f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:32.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 shutdown_connections 2024-04-27T05:21:32.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:32.942+0000 7fcae793c880 1 -- 172.21.15.138:0/2248185699 wait complete. 2024-04-27T05:21:32.949 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T05:21:32.950 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T05:21:32.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T05:21:32.950 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-27T05:21:33.036 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.036 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.034+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.034+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c4d53c880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b486e93000 0x55b487174580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c4d53c880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b48716a600 con 0x55b486e93400 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c4d53c880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b486ed2280 con 0x55b486e93000 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c49f3f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b486e93000 0x55b487174580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c49f3f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b486e93000 0x55b487174580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42138/0 (socket says 172.21.15.138:42138) 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c4973e640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55b486e93400 0x55b486f9a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60684/0 (socket says 172.21.15.138:60684) 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c49f3f640 1 -- 172.21.15.138:0/3135370175 learned_addr learned my addr 172.21.15.138:0/3135370175 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c49f3f640 1 -- 172.21.15.138:0/3135370175 >> v1:172.21.15.114:6789/0 conn(0x55b486e93400 legacy=0x55b486f9a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.035+0000 7f3c49f3f640 1 -- 172.21.15.138:0/3135370175 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486e97380 con 0x55b486e93000 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/3135370175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55b486f9ed50 tx=0x55b486fa4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab3f23e8cfdd4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c48f3d640 1 -- 172.21.15.138:0/3135370175 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabce0 con 0x55b486e93000 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c48f3d640 1 -- 172.21.15.138:0/3135370175 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b48716bb00 con 0x55b486e93000 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c48f3d640 1 -- 172.21.15.138:0/3135370175 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabe40 con 0x55b486e93000 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3135370175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 msgr2=0x55b487174580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3135370175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55b486f9ed50 tx=0x55b486fa4de0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3135370175 shutdown_connections 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3135370175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3135370175 >> 172.21.15.138:0/3135370175 conn(0x55b486e92c00 msgr2=0x55b486f9a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3135370175 shutdown_connections 2024-04-27T05:21:33.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.037+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3135370175 wait complete. 2024-04-27T05:21:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.038+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.038+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.038+0000 7f3c4d53c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.038+0000 7f3c4d53c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486ed2780 con 0x55b486e93000 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42152/0 (socket says 172.21.15.138:42152) 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c49f3f640 1 -- 172.21.15.138:0/2179173527 learned_addr learned my addr 172.21.15.138:0/2179173527 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c49f3f640 1 -- 172.21.15.138:0/2179173527 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486e96ea0 con 0x55b486e93000 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/2179173527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55b487c97560 tx=0x55b486fa4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5ff70bb92b0ada server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c4873c640 1 -- 172.21.15.138:0/2179173527 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eab4a0 con 0x55b486e93000 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c4873c640 1 -- 172.21.15.138:0/2179173527 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b48716be00 con 0x55b486e93000 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.039+0000 7f3c4873c640 1 -- 172.21.15.138:0/2179173527 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabce0 con 0x55b486e93000 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- 172.21.15.138:0/2179173527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 msgr2=0x55b487174580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/2179173527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55b487c97560 tx=0x55b486fa4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- 172.21.15.138:0/2179173527 shutdown_connections 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/2179173527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- 172.21.15.138:0/2179173527 >> 172.21.15.138:0/2179173527 conn(0x55b486e92c00 msgr2=0x55b486f9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- 172.21.15.138:0/2179173527 shutdown_connections 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- 172.21.15.138:0/2179173527 wait complete. 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c4d53c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486ed28c0 con 0x55b486e93000 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42162/0 (socket says 172.21.15.138:42162) 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.040+0000 7f3c49f3f640 1 -- 172.21.15.138:0/1499984448 learned_addr learned my addr 172.21.15.138:0/1499984448 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c49f3f640 1 -- 172.21.15.138:0/1499984448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486e976c0 con 0x55b486e93000 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55b487d4f2c0 tx=0x55b486fa4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c46f39640 1 -- 172.21.15.138:0/1499984448 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabe40 con 0x55b486e93000 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c46f39640 1 -- 172.21.15.138:0/1499984448 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b487c72000 con 0x55b486e93000 2024-04-27T05:21:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b486e971e0 con 0x55b486e93000 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c46f39640 1 -- 172.21.15.138:0/1499984448 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eab4a0 con 0x55b486e93000 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b486f04ea0 con 0x55b486e93000 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.041+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c46f39640 1 -- 172.21.15.138:0/1499984448 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b486ebcdc0 con 0x55b486e93000 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4d53c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4d53c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486ed2c80 con 0x55b487c8ac00 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c46f39640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487c8bc00 0x55b487175080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c46f39640 1 -- 172.21.15.138:0/1499984448 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b486ea5a40 con 0x55b486e93000 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4a740640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4a740640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42172/0 (socket says 172.21.15.138:42172) 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4a740640 1 -- 172.21.15.138:0/3880332278 learned_addr learned my addr 172.21.15.138:0/3880332278 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487c8bc00 0x55b487175080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4a740640 1 -- 172.21.15.138:0/3880332278 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486f04d00 con 0x55b487c8ac00 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c4a740640 1 --2- 172.21.15.138:0/3880332278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55b487c975f0 tx=0x55b486fa5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d341df6bc66c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487c8bc00 0x55b487175080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55b487d88780 tx=0x55b486fa4cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c44734640 1 -- 172.21.15.138:0/3880332278 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabe40 con 0x55b487c8ac00 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c44734640 1 -- 172.21.15.138:0/3880332278 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b487c72600 con 0x55b487c8ac00 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.042+0000 7f3c44734640 1 -- 172.21.15.138:0/3880332278 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eab4a0 con 0x55b487c8ac00 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3880332278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 msgr2=0x55b487174b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3880332278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55b487c975f0 tx=0x55b486fa5c20 comp rx=0 tx=0).stop 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3880332278 shutdown_connections 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3880332278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3880332278 >> 172.21.15.138:0/3880332278 conn(0x55b486e93400 msgr2=0x55b486f9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3880332278 shutdown_connections 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3880332278 wait complete. 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4d53c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486eb6280 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4a740640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4a740640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42174/0 (socket says 172.21.15.138:42174) 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.043+0000 7f3c4a740640 1 -- 172.21.15.138:0/3930579481 learned_addr learned my addr 172.21.15.138:0/3930579481 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c4a740640 1 -- 172.21.15.138:0/3930579481 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486f04b60 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c4a740640 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b487d4eb40 tx=0x55b486fa51a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c42f31640 1 -- 172.21.15.138:0/3930579481 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabe40 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c42f31640 1 -- 172.21.15.138:0/3930579481 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b487c72780 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b486f049c0 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c42f31640 1 -- 172.21.15.138:0/3930579481 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eab4a0 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.044+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b486f04820 con 0x55b487c8ac00 2024-04-27T05:21:33.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c42f31640 1 -- 172.21.15.138:0/3930579481 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b486ebcb00 con 0x55b487c8ac00 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c42f31640 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487210000 0x55b487175600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c42f31640 1 -- 172.21.15.138:0/3930579481 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b486ea5c00 con 0x55b487c8ac00 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4d53c880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4d53c880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4973e640 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487210000 0x55b487175600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4d53c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4d53c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486eb7180 con 0x55b487c8b400 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c4973e640 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487210000 0x55b487175600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55b486f9f380 tx=0x55b4871d6120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.045+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42184/0 (socket says 172.21.15.138:42184) 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c49f3f640 1 -- 172.21.15.138:0/4004958714 learned_addr learned my addr 172.21.15.138:0/4004958714 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c49f3f640 1 -- 172.21.15.138:0/4004958714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486f04680 con 0x55b487c8b400 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/4004958714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55b487d88900 tx=0x55b486fa4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf5dfbd50925d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c30f0d640 1 -- 172.21.15.138:0/4004958714 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabce0 con 0x55b487c8b400 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c30f0d640 1 -- 172.21.15.138:0/4004958714 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b487c72300 con 0x55b487c8b400 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c30f0d640 1 -- 172.21.15.138:0/4004958714 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabe40 con 0x55b487c8b400 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 -- 172.21.15.138:0/4004958714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 msgr2=0x55b487175b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/4004958714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55b487d88900 tx=0x55b486fa4f60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 -- 172.21.15.138:0/4004958714 shutdown_connections 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/4004958714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 -- 172.21.15.138:0/4004958714 >> 172.21.15.138:0/4004958714 conn(0x55b487c8b000 msgr2=0x55b486f9d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 -- 172.21.15.138:0/4004958714 shutdown_connections 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.046+0000 7f3c4d53c880 1 -- 172.21.15.138:0/4004958714 wait complete. 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c4d53c880 1 Processor -- start 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c4d53c880 1 -- start start 2024-04-27T05:21:33.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c4d53c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c4d53c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b486eb6b40 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c49f3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42198/0 (socket says 172.21.15.138:42198) 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c49f3f640 1 -- 172.21.15.138:0/3060021076 learned_addr learned my addr 172.21.15.138:0/3060021076 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c49f3f640 1 -- 172.21.15.138:0/3060021076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b486f044e0 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b4871f25a0 tx=0x55b486fa4ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.047+0000 7f3c2f70a640 1 -- 172.21.15.138:0/3060021076 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eab4a0 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c2f70a640 1 -- 172.21.15.138:0/3060021076 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b487c72480 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b486e97380 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c2f70a640 1 -- 172.21.15.138:0/3060021076 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b486eabce0 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b486e96ea0 con 0x55b487c8b400 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c2d706640 20 reqs_thread_entry: start 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b487244000 0x55b487176100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b487244400 con 0x55b487244000 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4973e640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b487244000 0x55b487176100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c2f70a640 1 -- 172.21.15.138:0/3060021076 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b486ebd8c0 con 0x55b487c8b400 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4973e640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b487244000 0x55b487176100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c2f70a640 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487245400 0x55b487176c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c2f70a640 1 -- 172.21.15.138:0/3060021076 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b486ea5dc0 con 0x55b487c8b400 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.048+0000 7f3c4a740640 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487245400 0x55b487176c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c4973e640 1 -- 172.21.15.138:0/1499984448 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b4872306c0 con 0x55b487244000 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b487244800 0x55b487176680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:2 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b487244c00 con 0x55b487244800 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b487244800 0x55b487176680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c4a740640 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487245400 0x55b487176c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55b487dff680 tx=0x55b4871d6a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c49f3f640 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b487244800 0x55b487176680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c49f3f640 1 -- 172.21.15.138:0/1499984448 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(2 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [call out=24b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+898 (crc 0 0 0) 0x55b487230900 con 0x55b487244800 2024-04-27T05:21:33.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.049+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [stat,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b487245000 con 0x55b487244800 2024-04-27T05:21:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.050+0000 7f3c49f3f640 1 -- 172.21.15.138:0/1499984448 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(3 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [stat,call,writefull 0~874] v29'2 uv2 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x55b487230900 con 0x55b487244800 2024-04-27T05:21:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487210000 msgr2=0x55b487175600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487210000 0x55b487175600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55b486f9f380 tx=0x55b4871d6120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 msgr2=0x55b487174b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55b487d4eb40 tx=0x55b486fa51a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 shutdown_connections 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487210000 0x55b487175600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3930579481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8ac00 0x55b487174b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 >> 172.21.15.138:0/3930579481 conn(0x55b486e93400 msgr2=0x55b486f9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 shutdown_connections 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3930579481 wait complete. 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487245400 msgr2=0x55b487176c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487245400 0x55b487176c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55b487dff680 tx=0x55b4871d6a20 comp rx=0 tx=0).stop 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 msgr2=0x55b487175b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55b4871f25a0 tx=0x55b486fa4ea0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 shutdown_connections 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487245400 0x55b487176c00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/3060021076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b487c8b400 0x55b487175b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.051+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 >> 172.21.15.138:0/3060021076 conn(0x55b487c8b000 msgr2=0x55b486f9d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 shutdown_connections 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/3060021076 wait complete. 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b487244000 msgr2=0x55b487176100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b487244000 0x55b487176100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b487244800 msgr2=0x55b487176680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b487244800 0x55b487176680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487c8bc00 msgr2=0x55b487175080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487c8bc00 0x55b487175080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55b487d88780 tx=0x55b486fa4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 msgr2=0x55b487174580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55b487d4f2c0 tx=0x55b486fa4c60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 shutdown_connections 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b487244800 0x55b487176680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b487244000 0x55b487176100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b487c8bc00 0x55b487175080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 --2- 172.21.15.138:0/1499984448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b486e93000 0x55b487174580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 >> 172.21.15.138:0/1499984448 conn(0x55b486e92c00 msgr2=0x55b486f9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 shutdown_connections 2024-04-27T05:21:33.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.052+0000 7f3c4d53c880 1 -- 172.21.15.138:0/1499984448 wait complete. 2024-04-27T05:21:33.067 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "control_pool": "default.rgw.control", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "log_pool": "default.rgw.log", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T05:21:33.068 INFO:teuthology.orchestra.run.smithi138.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "system_key": { 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "" 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_pools": [ 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "default-placement", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:21:33.069 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": true 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: "realm_id": "", 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T05:21:33.070 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:33.071 DEBUG:tasks.util.rgw: json result: {'id': '2f19b8ff-2608-4d86-8f1c-430623205476', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T05:21:33.072 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-27T05:21:33.072 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-27T05:21:33.072 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-27T05:21:33.113 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.113 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.111+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.111+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.111+0000 7f2bc5131880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55702a639000 0x55702a91a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.111+0000 7f2bc5131880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55702a910600 con 0x55702a639400 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.111+0000 7f2bc5131880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55702a678280 con 0x55702a639000 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.112+0000 7f2bc1b6f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55702a639000 0x55702a91a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.112+0000 7f2bc1b6f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55702a639000 0x55702a91a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42200/0 (socket says 172.21.15.138:42200) 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.112+0000 7f2bc136e640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55702a639400 0x55702a740800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60686/0 (socket says 172.21.15.138:60686) 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.112+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2906251437 learned_addr learned my addr 172.21.15.138:0/2906251437 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.112+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2906251437 >> v1:172.21.15.114:6789/0 conn(0x55702a639400 legacy=0x55702a740800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.112+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2906251437 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a63d380 con 0x55702a639000 2024-04-27T05:21:33.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.113+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/2906251437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55702a744d50 tx=0x55702a74ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1808a71f62b50a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc0b6d640 1 -- 172.21.15.138:0/2906251437 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a651ce0 con 0x55702a639000 2024-04-27T05:21:33.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc0b6d640 1 -- 172.21.15.138:0/2906251437 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702a911b00 con 0x55702a639000 2024-04-27T05:21:33.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc0b6d640 1 -- 172.21.15.138:0/2906251437 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a651e40 con 0x55702a639000 2024-04-27T05:21:33.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 -- 172.21.15.138:0/2906251437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 msgr2=0x55702a91a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2906251437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55702a744d50 tx=0x55702a74ade0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 -- 172.21.15.138:0/2906251437 shutdown_connections 2024-04-27T05:21:33.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2906251437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 -- 172.21.15.138:0/2906251437 >> 172.21.15.138:0/2906251437 conn(0x55702a638c00 msgr2=0x55702a740000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 -- 172.21.15.138:0/2906251437 shutdown_connections 2024-04-27T05:21:33.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.114+0000 7f2bc5131880 1 -- 172.21.15.138:0/2906251437 wait complete. 2024-04-27T05:21:33.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc5131880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc5131880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702a678780 con 0x55702a639000 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc1b6f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc1b6f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42216/0 (socket says 172.21.15.138:42216) 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2463489077 learned_addr learned my addr 172.21.15.138:0/2463489077 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2463489077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a63cea0 con 0x55702a639000 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.115+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/2463489077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55702b43d560 tx=0x55702a74acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848f82ebc17d317 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc036c640 1 -- 172.21.15.138:0/2463489077 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a6514a0 con 0x55702a639000 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc036c640 1 -- 172.21.15.138:0/2463489077 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702a911e00 con 0x55702a639000 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc036c640 1 -- 172.21.15.138:0/2463489077 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a651ce0 con 0x55702a639000 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- 172.21.15.138:0/2463489077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 msgr2=0x55702a91a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2463489077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55702b43d560 tx=0x55702a74acc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- 172.21.15.138:0/2463489077 shutdown_connections 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2463489077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- 172.21.15.138:0/2463489077 >> 172.21.15.138:0/2463489077 conn(0x55702a638c00 msgr2=0x55702a741000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- 172.21.15.138:0/2463489077 shutdown_connections 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- 172.21.15.138:0/2463489077 wait complete. 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.116+0000 7f2bc5131880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702a6788c0 con 0x55702a639000 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc1b6f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc1b6f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42228/0 (socket says 172.21.15.138:42228) 2024-04-27T05:21:33.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/3089952973 learned_addr learned my addr 172.21.15.138:0/3089952973 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/3089952973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a63d6c0 con 0x55702a639000 2024-04-27T05:21:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55702b4f52c0 tx=0x55702a74ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bbeb69640 1 -- 172.21.15.138:0/3089952973 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a651e40 con 0x55702a639000 2024-04-27T05:21:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bbeb69640 1 -- 172.21.15.138:0/3089952973 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702b418000 con 0x55702a639000 2024-04-27T05:21:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55702a63d1e0 con 0x55702a639000 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bbeb69640 1 -- 172.21.15.138:0/3089952973 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a6514a0 con 0x55702a639000 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.117+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702a6aaea0 con 0x55702a639000 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bbeb69640 1 -- 172.21.15.138:0/3089952973 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55702a662b00 con 0x55702a639000 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc5131880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc5131880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702a678c80 con 0x55702b430c00 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bbeb69640 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b431c00 0x55702a91b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bbeb69640 1 -- 172.21.15.138:0/3089952973 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55702a64ba40 con 0x55702a639000 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc2370640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc2370640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42230/0 (socket says 172.21.15.138:42230) 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc2370640 1 -- 172.21.15.138:0/3048368647 learned_addr learned my addr 172.21.15.138:0/3048368647 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b431c00 0x55702a91b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.118+0000 7f2bc2370640 1 -- 172.21.15.138:0/3048368647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a6aad00 con 0x55702b430c00 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b431c00 0x55702a91b080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55702b536780 tx=0x55702a74acc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc2370640 1 --2- 172.21.15.138:0/3048368647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55702b43d5f0 tx=0x55702a74bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b26575cf7cb3bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bbc364640 1 -- 172.21.15.138:0/3048368647 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a651e40 con 0x55702b430c00 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bbc364640 1 -- 172.21.15.138:0/3048368647 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702b418600 con 0x55702b430c00 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bbc364640 1 -- 172.21.15.138:0/3048368647 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a6514a0 con 0x55702b430c00 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 -- 172.21.15.138:0/3048368647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 msgr2=0x55702a91ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 --2- 172.21.15.138:0/3048368647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55702b43d5f0 tx=0x55702a74bc20 comp rx=0 tx=0).stop 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 -- 172.21.15.138:0/3048368647 shutdown_connections 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 --2- 172.21.15.138:0/3048368647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 -- 172.21.15.138:0/3048368647 >> 172.21.15.138:0/3048368647 conn(0x55702a639400 msgr2=0x55702a743000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 -- 172.21.15.138:0/3048368647 shutdown_connections 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 -- 172.21.15.138:0/3048368647 wait complete. 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.119+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc5131880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc5131880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702a65c280 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc2370640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc2370640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42242/0 (socket says 172.21.15.138:42242) 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 learned_addr learned my addr 172.21.15.138:0/2902523172 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a6aab60 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55702b4f4b40 tx=0x55702a74b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bbab61640 1 -- 172.21.15.138:0/2902523172 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a651e40 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55702a6aa9c0 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bbab61640 1 -- 172.21.15.138:0/2902523172 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702b418780 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702a6aa820 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.120+0000 7f2bbab61640 1 -- 172.21.15.138:0/2902523172 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a6514a0 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bbab61640 1 -- 172.21.15.138:0/2902523172 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55702a663600 con 0x55702b430c00 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bbab61640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702a9be000 0x55702a91b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bbab61640 1 -- 172.21.15.138:0/2902523172 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55702a64bc00 con 0x55702b430c00 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bc136e640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702a9be000 0x55702a91b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55702b431000 0x55702a91bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.121+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b431400 con 0x55702b431000 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55702b431000 0x55702a91bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc136e640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702a9be000 0x55702a91b600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55702a745380 tx=0x55702a97c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55702b431000 0x55702a91bb80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 realm 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b431800 con 0x55702b431000 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bf000 con 0x55702b431000 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.122+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bf400 con 0x55702b431000 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55702a9bf800 0x55702a91c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bfc00 con 0x55702a9bf800 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55702a9bf800 0x55702a91c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55702a9bf800 0x55702a91c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 20 searching for the correct realm 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.123+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bf000 con 0x55702a9bf800 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc1b6f640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55702a9bf000 0x55702a91cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bfc00 con 0x55702a9bf000 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc136e640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55702a9bf000 0x55702a91cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc136e640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55702a9bf000 0x55702a91cc00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc136e640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55702a9bfc00 0x55702a91d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.124+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.125+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55702a9bfc00 0x55702a91d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.125+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55702a9bfc00 0x55702a91d180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.125+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.125+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf000 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.126+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.127+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf000 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.128+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.129+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf000 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.130+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf000 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf800 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.131+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bfc00 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702b431000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bec00 con 0x55702a9bf000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bf400 con 0x55702b431000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.132+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b431400 con 0x55702b431000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b431800 con 0x55702a9bf800 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 zone default found 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 4 Realm: () 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 zonegroup default 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de000 con 0x55702a9bfc00 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de400 con 0x55702a9bf800 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.133+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de800 con 0x55702a9bf800 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.134+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.134+0000 7f2bc5131880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.134+0000 7f2bc5131880 20 started sync module instance, tier type = 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.134+0000 7f2bc5131880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.134+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9dec00 con 0x55702a9bf000 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.135+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.135+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93940238854400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9df000 con 0x55702a9bf000 2024-04-27T05:21:33.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.135+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9df400 con 0x55702b431000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.136+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.136+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93940238855552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9df800 con 0x55702b431000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.136+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9dfc00 con 0x55702a9bf800 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.137+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93940238855552] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.137+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93940238854400] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.137+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.138+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93940238856704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de400 con 0x55702a9bf800 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.138+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de800 con 0x55702a9bf000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.138+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93940238856704] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.139+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.139+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93940238857856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9dfc00 con 0x55702a9bf000 2024-04-27T05:21:33.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.139+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de000 con 0x55702b431000 2024-04-27T05:21:33.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.140+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93940238857856] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.140+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.140+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93940238859008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b431800 con 0x55702b431000 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.140+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b431400 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.140+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.141+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93940238860160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9bf400 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.141+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b597800 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.141+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93940238859008] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.141+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93940238860160] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.141+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93940238861312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b597400 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9dec00 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93940238862464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9df000 con 0x55702a9bfc00 2024-04-27T05:21:33.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=0 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=1 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=2 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=3 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=4 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=5 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93940238861312] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.142+0000 7f2bc5131880 20 add_watcher() i=6 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.143+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93940238862464] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.143+0000 7f2bc5131880 20 add_watcher() i=7 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.143+0000 7f2bc5131880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.143+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de800 con 0x55702b431000 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.144+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.144+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93940238863616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9df400 con 0x55702b431000 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93940238863616] v29'52 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2ba8b3d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc5131880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc5131880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc5131880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc5131880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702a65c8c0 con 0x55702a9bec00 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc136e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc136e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42248/0 (socket says 172.21.15.138:42248) 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.145+0000 7f2bc136e640 1 -- 172.21.15.138:0/624475101 learned_addr learned my addr 172.21.15.138:0/624475101 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.146+0000 7f2bc136e640 1 -- 172.21.15.138:0/624475101 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a6aa680 con 0x55702a9bec00 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.146+0000 7f2bc136e640 1 --2- 172.21.15.138:0/624475101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55702a99eae0 tx=0x55702a97c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276a776d5fe3f83c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.146+0000 7f2ba833c640 1 -- 172.21.15.138:0/624475101 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a9e0580 con 0x55702a9bec00 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.146+0000 7f2ba833c640 1 -- 172.21.15.138:0/624475101 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702b418000 con 0x55702a9bec00 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.146+0000 7f2ba833c640 1 -- 172.21.15.138:0/624475101 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a9e06e0 con 0x55702a9bec00 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- 172.21.15.138:0/624475101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 msgr2=0x55702a91c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 --2- 172.21.15.138:0/624475101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55702a99eae0 tx=0x55702a97c180 comp rx=0 tx=0).stop 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- 172.21.15.138:0/624475101 shutdown_connections 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 --2- 172.21.15.138:0/624475101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- 172.21.15.138:0/624475101 >> 172.21.15.138:0/624475101 conn(0x55702a9df800 msgr2=0x55702a743800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- 172.21.15.138:0/624475101 shutdown_connections 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- 172.21.15.138:0/624475101 wait complete. 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 Processor -- start 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- start start 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc5131880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702a65c780 con 0x55702a9bec00 2024-04-27T05:21:33.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc136e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc136e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42260/0 (socket says 172.21.15.138:42260) 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc136e640 1 -- 172.21.15.138:0/2969261813 learned_addr learned my addr 172.21.15.138:0/2969261813 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.147+0000 7f2bc136e640 1 -- 172.21.15.138:0/2969261813 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702a6aa4e0 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2bc136e640 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55702a99f860 tx=0x55702a97cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2ba6b39640 1 -- 172.21.15.138:0/2969261813 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a9e0840 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2ba6b39640 1 -- 172.21.15.138:0/2969261813 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55702b418a80 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55702a63d380 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2ba6b39640 1 -- 172.21.15.138:0/2969261813 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55702a9e0580 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702a63cea0 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.148+0000 7f2ba6b39640 1 -- 172.21.15.138:0/2969261813 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55702a663b80 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2ba6b39640 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b597800 0x55702a91d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2ba6b39640 1 -- 172.21.15.138:0/2969261813 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55702a64bdc0 con 0x55702a9bec00 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc5131880 5 note: GC not initialized 2024-04-27T05:21:33.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b597800 0x55702a91d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2ba4334640 20 reqs_thread_entry: start 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc5131880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc2370640 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b597800 0x55702a91d700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55702b590c30 tx=0x55702a97c8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc5131880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2ba2330640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:03 2024 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2ba1b0e640 20 reqs_thread_entry: start 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac000 con 0x55702b431000 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.149+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac400 con 0x55702a9bf000 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac800 con 0x55702b431000 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaacc00 con 0x55702a9bf800 2024-04-27T05:21:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad000 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.150+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad400 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad800 con 0x55702b431000 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaadc00 con 0x55702b431000 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8000 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8400 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.151+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8800 con 0x55702a9bfc00 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8c00 con 0x55702b431000 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9000 con 0x55702a9bf000 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9400 con 0x55702a9bf800 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.152+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9800 con 0x55702a9bfc00 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9c00 con 0x55702a9bfc00 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8000 con 0x55702a9bf800 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8400 con 0x55702a9bfc00 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.153+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9400 con 0x55702b431000 2024-04-27T05:21:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8800 con 0x55702a9bf800 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaacc00 con 0x55702a9bfc00 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad000 con 0x55702a9bf000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.154+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad400 con 0x55702a9bfc00 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac000 con 0x55702b431000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac800 con 0x55702b431000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad800 con 0x55702a9bf800 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaadc00 con 0x55702b431000 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9de400 con 0x55702b431000 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.155+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9dec00 con 0x55702a9bf800 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.156+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.156+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702a9df000 con 0x55702a9bf800 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.156+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.156+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaacc00 con 0x55702a9bf800 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.156+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.156+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaad400 con 0x55702a9bfc00 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.157+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.157+0000 7f2ba8b3d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.157+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93940238863616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac000 con 0x55702b431000 2024-04-27T05:21:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.158+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93940238863616] v29'53 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.158+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93940238854400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8c00 con 0x55702a9bf000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.159+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93940238854400] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.159+0000 7f2bc5131880 20 remove_watcher() i=0 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.159+0000 7f2bc5131880 2 removed watcher, disabling cache 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.159+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93940238855552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9800 con 0x55702b431000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.160+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93940238855552] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.160+0000 7f2bc5131880 20 remove_watcher() i=1 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.160+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93940238856704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9c00 con 0x55702a9bf800 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.161+0000 7f2bc1b6f640 1 -- 172.21.15.138:0/2902523172 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93940238856704] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cafc0 con 0x55702a9bf800 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.161+0000 7f2bc5131880 20 remove_watcher() i=2 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.162+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93940238857856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8400 con 0x55702a9bf000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.162+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93940238857856] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9cb200 con 0x55702a9bf000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.162+0000 7f2bc5131880 20 remove_watcher() i=3 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.162+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93940238859008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9400 con 0x55702b431000 2024-04-27T05:21:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.163+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93940238859008] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702b431000 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.163+0000 7f2bc5131880 20 remove_watcher() i=4 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.163+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93940238860160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab9000 con 0x55702a9bfc00 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.164+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93940238860160] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.164+0000 7f2bc5131880 20 remove_watcher() i=5 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.164+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93940238861312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aab8c00 con 0x55702a9bfc00 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.165+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93940238861312] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.165+0000 7f2bc5131880 20 remove_watcher() i=6 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.165+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93940238862464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702aaac400 con 0x55702a9bfc00 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc2370640 1 -- 172.21.15.138:0/2902523172 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93940238862464] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55702a9ca900 con 0x55702a9bfc00 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 20 remove_watcher() i=7 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55702a9bfc00 msgr2=0x55702a91d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55702a9bfc00 0x55702a91d180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55702b431000 msgr2=0x55702a91bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55702b431000 0x55702a91bb80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55702a9bf000 msgr2=0x55702a91cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55702a9bf000 0x55702a91cc00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55702a9bf800 msgr2=0x55702a91c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55702a9bf800 0x55702a91c100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702a9be000 msgr2=0x55702a91b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702a9be000 0x55702a91b600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55702a745380 tx=0x55702a97c120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 msgr2=0x55702a91ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2902523172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702b430c00 0x55702a91ab00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55702b4f4b40 tx=0x55702a74b1a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 reap_dead start 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc136e640 1 -- 172.21.15.138:0/2902523172 reap_dead start 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 shutdown_connections 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 >> 172.21.15.138:0/2902523172 conn(0x55702a639400 msgr2=0x55702a743000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 shutdown_connections 2024-04-27T05:21:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.166+0000 7f2bc5131880 1 -- 172.21.15.138:0/2902523172 wait complete. 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b597800 msgr2=0x55702a91d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b597800 0x55702a91d700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55702b590c30 tx=0x55702a97c8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 msgr2=0x55702a91c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55702a99f860 tx=0x55702a97cae0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 shutdown_connections 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b597800 0x55702a91d700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/2969261813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a9bec00 0x55702a91c680 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 >> 172.21.15.138:0/2969261813 conn(0x55702a9df800 msgr2=0x55702a743800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 shutdown_connections 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/2969261813 wait complete. 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b431c00 msgr2=0x55702a91b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b431c00 0x55702a91b080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55702b536780 tx=0x55702a74acc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 msgr2=0x55702a91a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55702b4f52c0 tx=0x55702a74ac60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 shutdown_connections 2024-04-27T05:21:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55702b431c00 0x55702a91b080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 --2- 172.21.15.138:0/3089952973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55702a639000 0x55702a91a580 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 >> 172.21.15.138:0/3089952973 conn(0x55702a638c00 msgr2=0x55702a741000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 shutdown_connections 2024-04-27T05:21:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.167+0000 7f2bc5131880 1 -- 172.21.15.138:0/3089952973 wait complete. 2024-04-27T05:21:33.189 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:33.189 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [], 2024-04-27T05:21:33.189 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [] 2024-04-27T05:21:33.189 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:33.189 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-27T05:21:33.189 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:33.189 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:33.189 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:33.271 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.271 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.269+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.269+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.269+0000 7f39636bb880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.269+0000 7f39636bb880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d70d694600 con 0x55d70d3bd400 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.269+0000 7f39636bb880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d70d3fc280 con 0x55d70d3bd000 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.270+0000 7f3961e60640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.270+0000 7f3961e60640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42272/0 (socket says 172.21.15.138:42272) 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.270+0000 7f3961e60640 1 -- 172.21.15.138:0/4183229655 learned_addr learned my addr 172.21.15.138:0/4183229655 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.270+0000 7f396165f640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55d70d3bd400 0x55d70d4c4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60698/0 (socket says 172.21.15.138:60698) 2024-04-27T05:21:33.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.270+0000 7f3961e60640 1 -- 172.21.15.138:0/4183229655 >> v1:172.21.15.114:6789/0 conn(0x55d70d3bd400 legacy=0x55d70d4c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.270+0000 7f3961e60640 1 -- 172.21.15.138:0/4183229655 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d3c1380 con 0x55d70d3bd000 2024-04-27T05:21:33.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f3961e60640 1 --2- 172.21.15.138:0/4183229655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55d70d4c8d50 tx=0x55d70d4cede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c2eec5f8895e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f3960e5e640 1 -- 172.21.15.138:0/4183229655 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d5ce0 con 0x55d70d3bd000 2024-04-27T05:21:33.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f3960e5e640 1 -- 172.21.15.138:0/4183229655 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70d695b00 con 0x55d70d3bd000 2024-04-27T05:21:33.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f3960e5e640 1 -- 172.21.15.138:0/4183229655 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d5e40 con 0x55d70d3bd000 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 -- 172.21.15.138:0/4183229655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 msgr2=0x55d70d69e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 --2- 172.21.15.138:0/4183229655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55d70d4c8d50 tx=0x55d70d4cede0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 -- 172.21.15.138:0/4183229655 shutdown_connections 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 --2- 172.21.15.138:0/4183229655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 -- 172.21.15.138:0/4183229655 >> 172.21.15.138:0/4183229655 conn(0x55d70d3bcc00 msgr2=0x55d70d4c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 -- 172.21.15.138:0/4183229655 shutdown_connections 2024-04-27T05:21:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.272+0000 7f39636bb880 1 -- 172.21.15.138:0/4183229655 wait complete. 2024-04-27T05:21:33.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f39636bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f39636bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70d3fc780 con 0x55d70d3bd000 2024-04-27T05:21:33.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f3961e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f3961e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42288/0 (socket says 172.21.15.138:42288) 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.273+0000 7f3961e60640 1 -- 172.21.15.138:0/2083991469 learned_addr learned my addr 172.21.15.138:0/2083991469 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f3961e60640 1 -- 172.21.15.138:0/2083991469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d3c0ea0 con 0x55d70d3bd000 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f3961e60640 1 --2- 172.21.15.138:0/2083991469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55d70e1c1560 tx=0x55d70d4cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53a47a795862397 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f396065d640 1 -- 172.21.15.138:0/2083991469 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d54a0 con 0x55d70d3bd000 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f396065d640 1 -- 172.21.15.138:0/2083991469 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70d695e00 con 0x55d70d3bd000 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f396065d640 1 -- 172.21.15.138:0/2083991469 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d5ce0 con 0x55d70d3bd000 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 -- 172.21.15.138:0/2083991469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 msgr2=0x55d70d69e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 --2- 172.21.15.138:0/2083991469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55d70e1c1560 tx=0x55d70d4cecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 -- 172.21.15.138:0/2083991469 shutdown_connections 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 --2- 172.21.15.138:0/2083991469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 -- 172.21.15.138:0/2083991469 >> 172.21.15.138:0/2083991469 conn(0x55d70d3bcc00 msgr2=0x55d70d4c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 -- 172.21.15.138:0/2083991469 shutdown_connections 2024-04-27T05:21:33.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.274+0000 7f39636bb880 1 -- 172.21.15.138:0/2083991469 wait complete. 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f39636bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f39636bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70d3fc8c0 con 0x55d70d3bd000 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f3961e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f3961e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42292/0 (socket says 172.21.15.138:42292) 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f3961e60640 1 -- 172.21.15.138:0/158456496 learned_addr learned my addr 172.21.15.138:0/158456496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f3961e60640 1 -- 172.21.15.138:0/158456496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d3c16c0 con 0x55d70d3bd000 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f3961e60640 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55d70e2792c0 tx=0x55d70d4cec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f395ee5a640 1 -- 172.21.15.138:0/158456496 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d5e40 con 0x55d70d3bd000 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f395ee5a640 1 -- 172.21.15.138:0/158456496 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70e27a000 con 0x55d70d3bd000 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d70d3c11e0 con 0x55d70d3bd000 2024-04-27T05:21:33.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.275+0000 7f395ee5a640 1 -- 172.21.15.138:0/158456496 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d54a0 con 0x55d70d3bd000 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d70d42eea0 con 0x55d70d3bd000 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f39636bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f39636bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70d3fcc80 con 0x55d70e1b4c00 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f395ee5a640 1 -- 172.21.15.138:0/158456496 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d70d3e6dc0 con 0x55d70d3bd000 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f3962661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f395ee5a640 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e1b5c00 0x55d70d69f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f3962661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42306/0 (socket says 172.21.15.138:42306) 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f3962661640 1 -- 172.21.15.138:0/4205569003 learned_addr learned my addr 172.21.15.138:0/4205569003 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.276+0000 7f395ee5a640 1 -- 172.21.15.138:0/158456496 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55d70d3cfa40 con 0x55d70d3bd000 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f3961e60640 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e1b5c00 0x55d70d69f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f3962661640 1 -- 172.21.15.138:0/4205569003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d42ed00 con 0x55d70e1b4c00 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f3962661640 1 --2- 172.21.15.138:0/4205569003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55d70e1c15f0 tx=0x55d70d4cee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b8b30459006633 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f3961e60640 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e1b5c00 0x55d70d69f080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55d70e2ac7b0 tx=0x55d70d4cef60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f395c655640 1 -- 172.21.15.138:0/4205569003 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d5e40 con 0x55d70e1b4c00 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f395c655640 1 -- 172.21.15.138:0/4205569003 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70e27a600 con 0x55d70e1b4c00 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f395c655640 1 -- 172.21.15.138:0/4205569003 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d54a0 con 0x55d70e1b4c00 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 -- 172.21.15.138:0/4205569003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 msgr2=0x55d70d69eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 --2- 172.21.15.138:0/4205569003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55d70e1c15f0 tx=0x55d70d4cee40 comp rx=0 tx=0).stop 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 -- 172.21.15.138:0/4205569003 shutdown_connections 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 --2- 172.21.15.138:0/4205569003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 -- 172.21.15.138:0/4205569003 >> 172.21.15.138:0/4205569003 conn(0x55d70d3bd400 msgr2=0x55d70d4c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 -- 172.21.15.138:0/4205569003 shutdown_connections 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.277+0000 7f39636bb880 1 -- 172.21.15.138:0/4205569003 wait complete. 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f39636bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f39636bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70d3e0280 con 0x55d70e1b4c00 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f3962661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f3962661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42308/0 (socket says 172.21.15.138:42308) 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 learned_addr learned my addr 172.21.15.138:0/2606188559 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d42eb60 con 0x55d70e1b4c00 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f3962661640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55d70e2793b0 tx=0x55d70d4cf9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.278+0000 7f395ae52640 1 -- 172.21.15.138:0/2606188559 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d5e40 con 0x55d70e1b4c00 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.279+0000 7f395ae52640 1 -- 172.21.15.138:0/2606188559 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70e27a780 con 0x55d70e1b4c00 2024-04-27T05:21:33.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.279+0000 7f395ae52640 1 -- 172.21.15.138:0/2606188559 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d3d54a0 con 0x55d70e1b4c00 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.279+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d70d42e9c0 con 0x55d70e1b4c00 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.279+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d70d42e820 con 0x55d70e1b4c00 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.279+0000 7f395ae52640 1 -- 172.21.15.138:0/2606188559 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d70d3e6b00 con 0x55d70e1b4c00 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f395ae52640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70d73a000 0x55d70d69f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f395ae52640 1 -- 172.21.15.138:0/2606188559 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55d70d3cfc00 con 0x55d70e1b4c00 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f396165f640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70d73a000 0x55d70d69f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d70e1b5000 0x55d70d69fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e1b5400 con 0x55d70e1b5000 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f3962661640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d70e1b5000 0x55d70d69fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f396165f640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70d73a000 0x55d70d69f600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55d70d4c9380 tx=0x55d70d700120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f3962661640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d70e1b5000 0x55d70d69fb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 20 realm 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.280+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e1b5800 con 0x55d70e1b5000 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73b000 con 0x55d70e1b5000 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73b400 con 0x55d70e1b5000 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d70d73b800 0x55d70d6a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73bc00 con 0x55d70d73b800 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f3961e60640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d70d73b800 0x55d70d6a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.281+0000 7f3961e60640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d70d73b800 0x55d70d6a0100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f39636bb880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f39636bb880 20 searching for the correct realm 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73b000 con 0x55d70d73b800 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f3961e60640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d70d73b000 0x55d70d6a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73bc00 con 0x55d70d73b000 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f396165f640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d70d73b000 0x55d70d6a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.282+0000 7f396165f640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d70d73b000 0x55d70d6a0c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.283+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.283+0000 7f396165f640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d70d73bc00 0x55d70d6a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.283+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.283+0000 7f3962661640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d70d73bc00 0x55d70d6a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.283+0000 7f3962661640 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d70d73bc00 0x55d70d6a1180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.284+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b000 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.285+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b000 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.286+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.287+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b000 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.288+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b000 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b800 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.289+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73bc00 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70e1b5000 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73ac00 con 0x55d70d73b000 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f39636bb880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f39636bb880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f39636bb880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f39636bb880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73b400 con 0x55d70e1b5000 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.290+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e1b5400 con 0x55d70e1b5000 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e1b5800 con 0x55d70d73b800 2024-04-27T05:21:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 zone default found 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 4 Realm: () 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 zonegroup default 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766000 con 0x55d70d73bc00 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766400 con 0x55d70d73b800 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.291+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766800 con 0x55d70d73b800 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.292+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.292+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.292+0000 7f39636bb880 20 started sync module instance, tier type = 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.292+0000 7f39636bb880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.292+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766c00 con 0x55d70d73b000 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.293+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.293+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94382131341568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767000 con 0x55d70d73b000 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.293+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767400 con 0x55d70e1b5000 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.294+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94382131341568] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.294+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.294+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94382131342720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767800 con 0x55d70e1b5000 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.294+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767c00 con 0x55d70d73b800 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.295+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94382131342720] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.295+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.295+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94382131343872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766400 con 0x55d70d73b800 2024-04-27T05:21:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.295+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766800 con 0x55d70d73b000 2024-04-27T05:21:33.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.296+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94382131343872] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.296+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.296+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94382131345024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767c00 con 0x55d70d73b000 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.296+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766000 con 0x55d70e1b5000 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.297+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94382131345024] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.297+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.297+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94382131346176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e1b5800 con 0x55d70e1b5000 2024-04-27T05:21:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.297+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e1b5400 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.298+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.298+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94382131346176] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.298+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94382131347328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d73b400 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.298+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e31b800 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.299+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.299+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94382131348480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70e31b400 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.299+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766c00 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.299+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94382131347328] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94382131349632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767000 con 0x55d70d73bc00 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=0 2024-04-27T05:21:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=1 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=2 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=3 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=4 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=5 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94382131348480] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.300+0000 7f39636bb880 20 add_watcher() i=6 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.301+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94382131349632] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.301+0000 7f39636bb880 20 add_watcher() i=7 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.301+0000 7f39636bb880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.301+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d766800 con 0x55d70e1b5000 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.301+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.301+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94382131350784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d767400 con 0x55d70e1b5000 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94382131350784] v29'54 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f3948e2e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f39636bb880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f39636bb880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f39636bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.302+0000 7f39636bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70d3e08c0 con 0x55d70d73ac00 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f396165f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f396165f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42324/0 (socket says 172.21.15.138:42324) 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f396165f640 1 -- 172.21.15.138:0/1148833126 learned_addr learned my addr 172.21.15.138:0/1148833126 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f396165f640 1 -- 172.21.15.138:0/1148833126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d42e680 con 0x55d70d73ac00 2024-04-27T05:21:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f396165f640 1 --2- 172.21.15.138:0/1148833126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d70d71cae0 tx=0x55d70d700180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479e7dca4e300a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f394862d640 1 -- 172.21.15.138:0/1148833126 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d768580 con 0x55d70d73ac00 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f394862d640 1 -- 172.21.15.138:0/1148833126 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70e27a000 con 0x55d70d73ac00 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.303+0000 7f394862d640 1 -- 172.21.15.138:0/1148833126 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d7686e0 con 0x55d70d73ac00 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- 172.21.15.138:0/1148833126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 msgr2=0x55d70d6a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 --2- 172.21.15.138:0/1148833126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d70d71cae0 tx=0x55d70d700180 comp rx=0 tx=0).stop 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- 172.21.15.138:0/1148833126 shutdown_connections 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 --2- 172.21.15.138:0/1148833126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- 172.21.15.138:0/1148833126 >> 172.21.15.138:0/1148833126 conn(0x55d70d767800 msgr2=0x55d70d4c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- 172.21.15.138:0/1148833126 shutdown_connections 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- 172.21.15.138:0/1148833126 wait complete. 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 Processor -- start 2024-04-27T05:21:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- start start 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f39636bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70d3e0780 con 0x55d70d73ac00 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f396165f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f396165f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42326/0 (socket says 172.21.15.138:42326) 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.304+0000 7f396165f640 1 -- 172.21.15.138:0/3787095966 learned_addr learned my addr 172.21.15.138:0/3787095966 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f396165f640 1 -- 172.21.15.138:0/3787095966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70d42e4e0 con 0x55d70d73ac00 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f396165f640 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d70d71d860 tx=0x55d70d700ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f3946e2a640 1 -- 172.21.15.138:0/3787095966 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d768840 con 0x55d70d73ac00 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f3946e2a640 1 -- 172.21.15.138:0/3787095966 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d70e27aa80 con 0x55d70d73ac00 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d70d3c1380 con 0x55d70d73ac00 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f3946e2a640 1 -- 172.21.15.138:0/3787095966 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d70d768580 con 0x55d70d73ac00 2024-04-27T05:21:33.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.305+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d70d3c0ea0 con 0x55d70d73ac00 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3946e2a640 1 -- 172.21.15.138:0/3787095966 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d70d3e78c0 con 0x55d70d73ac00 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3946e2a640 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e31b800 0x55d70d6a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3946e2a640 1 -- 172.21.15.138:0/3787095966 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55d70d3cfdc0 con 0x55d70d73ac00 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f39636bb880 5 note: GC not initialized 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3962661640 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e31b800 0x55d70d6a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3944625640 20 reqs_thread_entry: start 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f39636bb880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3962661640 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e31b800 0x55d70d6a1700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d70e328c30 tx=0x55d70d7008a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f39636bb880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3942621640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:03 2024 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f3941dff640 20 reqs_thread_entry: start 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f39636bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.306+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d836000 con 0x55d70e1b5000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.307+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.307+0000 7f39636bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:could not fetch user info: no user info saved 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.307+0000 7f3948e2e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.307+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94382131350784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d836400 con 0x55d70e1b5000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.308+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94382131350784] v29'55 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.308+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94382131341568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d836800 con 0x55d70d73b000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.309+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94382131341568] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.309+0000 7f39636bb880 20 remove_watcher() i=0 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.309+0000 7f39636bb880 2 removed watcher, disabling cache 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.309+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94382131342720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d836c00 con 0x55d70e1b5000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.310+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94382131342720] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.310+0000 7f39636bb880 20 remove_watcher() i=1 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.310+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94382131343872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d837000 con 0x55d70d73b800 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.311+0000 7f3961e60640 1 -- 172.21.15.138:0/2606188559 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94382131343872] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754fc0 con 0x55d70d73b800 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.311+0000 7f39636bb880 20 remove_watcher() i=2 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.311+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94382131345024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d837400 con 0x55d70d73b000 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.312+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94382131345024] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d755200 con 0x55d70d73b000 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.312+0000 7f39636bb880 20 remove_watcher() i=3 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.312+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94382131346176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d837800 con 0x55d70e1b5000 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.313+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94382131346176] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70e1b5000 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.313+0000 7f39636bb880 20 remove_watcher() i=4 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.313+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94382131347328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d837c00 con 0x55d70d73bc00 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.314+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94382131347328] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.314+0000 7f39636bb880 20 remove_watcher() i=5 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.314+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94382131348480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d836000 con 0x55d70d73bc00 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.315+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94382131348480] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.315+0000 7f39636bb880 20 remove_watcher() i=6 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.315+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94382131349632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d70d836400 con 0x55d70d73bc00 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f3962661640 1 -- 172.21.15.138:0/2606188559 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94382131349632] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d70d754900 con 0x55d70d73bc00 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 20 remove_watcher() i=7 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d70d73bc00 msgr2=0x55d70d6a1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d70d73bc00 0x55d70d6a1180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d70e1b5000 msgr2=0x55d70d69fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d70e1b5000 0x55d70d69fb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d70d73b000 msgr2=0x55d70d6a0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d70d73b000 0x55d70d6a0c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d70d73b800 msgr2=0x55d70d6a0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d70d73b800 0x55d70d6a0100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70d73a000 msgr2=0x55d70d69f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70d73a000 0x55d70d69f600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55d70d4c9380 tx=0x55d70d700120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 msgr2=0x55d70d69eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 --2- 172.21.15.138:0/2606188559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70e1b4c00 0x55d70d69eb00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55d70e2793b0 tx=0x55d70d4cf9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 reap_dead start 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f396165f640 1 -- 172.21.15.138:0/2606188559 reap_dead start 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 shutdown_connections 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 >> 172.21.15.138:0/2606188559 conn(0x55d70d3bd400 msgr2=0x55d70d4c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.316+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 shutdown_connections 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/2606188559 wait complete. 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e31b800 msgr2=0x55d70d6a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e31b800 0x55d70d6a1700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d70e328c30 tx=0x55d70d7008a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 msgr2=0x55d70d6a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d70d71d860 tx=0x55d70d700ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 shutdown_connections 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e31b800 0x55d70d6a1700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/3787095966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d73ac00 0x55d70d6a0680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 >> 172.21.15.138:0/3787095966 conn(0x55d70d767800 msgr2=0x55d70d4c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 shutdown_connections 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/3787095966 wait complete. 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e1b5c00 msgr2=0x55d70d69f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e1b5c00 0x55d70d69f080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55d70e2ac7b0 tx=0x55d70d4cef60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 msgr2=0x55d70d69e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55d70e2792c0 tx=0x55d70d4cec60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 shutdown_connections 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d70e1b5c00 0x55d70d69f080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 --2- 172.21.15.138:0/158456496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d70d3bd000 0x55d70d69e580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 >> 172.21.15.138:0/158456496 conn(0x55d70d3bcc00 msgr2=0x55d70d4c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 shutdown_connections 2024-04-27T05:21:33.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.317+0000 7f39636bb880 1 -- 172.21.15.138:0/158456496 wait complete. 2024-04-27T05:21:33.342 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-27T05:21:33.342 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-27T05:21:33.342 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-04-27T05:21:33.422 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.422 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.420+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.420+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.420+0000 7f144c536880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.420+0000 7f144c536880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5581a27e6600 con 0x5581a250f400 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.420+0000 7f144c536880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5581a254e280 con 0x5581a250f000 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.421+0000 7f1448e85640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.421+0000 7f1448684640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5581a250f400 0x5581a2616800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60706/0 (socket says 172.21.15.138:60706) 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.421+0000 7f1448e85640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42332/0 (socket says 172.21.15.138:42332) 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.421+0000 7f1448684640 1 -- 172.21.15.138:0/4112384546 learned_addr learned my addr 172.21.15.138:0/4112384546 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.421+0000 7f1448e85640 1 -- 172.21.15.138:0/4112384546 >> v1:172.21.15.114:6789/0 conn(0x5581a250f400 legacy=0x5581a2616800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.421+0000 7f1448e85640 1 -- 172.21.15.138:0/4112384546 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a2513380 con 0x5581a250f000 2024-04-27T05:21:33.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.422+0000 7f1448e85640 1 --2- 172.21.15.138:0/4112384546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5581a261ad50 tx=0x5581a2620de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66987649acd4f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f1447e83640 1 -- 172.21.15.138:0/4112384546 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a2527ce0 con 0x5581a250f000 2024-04-27T05:21:33.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f1447e83640 1 -- 172.21.15.138:0/4112384546 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a27e7b00 con 0x5581a250f000 2024-04-27T05:21:33.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f1447e83640 1 -- 172.21.15.138:0/4112384546 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a2527e40 con 0x5581a250f000 2024-04-27T05:21:33.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 -- 172.21.15.138:0/4112384546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 msgr2=0x5581a27f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 --2- 172.21.15.138:0/4112384546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5581a261ad50 tx=0x5581a2620de0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 -- 172.21.15.138:0/4112384546 shutdown_connections 2024-04-27T05:21:33.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 --2- 172.21.15.138:0/4112384546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 -- 172.21.15.138:0/4112384546 >> 172.21.15.138:0/4112384546 conn(0x5581a250ec00 msgr2=0x5581a2616000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 -- 172.21.15.138:0/4112384546 shutdown_connections 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.423+0000 7f144c536880 1 -- 172.21.15.138:0/4112384546 wait complete. 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f144c536880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f144c536880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a254e780 con 0x5581a250f000 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f1448e85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f1448e85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42344/0 (socket says 172.21.15.138:42344) 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f1448e85640 1 -- 172.21.15.138:0/3346953986 learned_addr learned my addr 172.21.15.138:0/3346953986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f1448e85640 1 -- 172.21.15.138:0/3346953986 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a2512ea0 con 0x5581a250f000 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.424+0000 7f1448e85640 1 --2- 172.21.15.138:0/3346953986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5581a3313560 tx=0x5581a2620cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb446ec2c6a028f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f1447682640 1 -- 172.21.15.138:0/3346953986 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a25274a0 con 0x5581a250f000 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f1447682640 1 -- 172.21.15.138:0/3346953986 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a27e7e00 con 0x5581a250f000 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f1447682640 1 -- 172.21.15.138:0/3346953986 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a2527ce0 con 0x5581a250f000 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- 172.21.15.138:0/3346953986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 msgr2=0x5581a27f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 --2- 172.21.15.138:0/3346953986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5581a3313560 tx=0x5581a2620cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- 172.21.15.138:0/3346953986 shutdown_connections 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 --2- 172.21.15.138:0/3346953986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- 172.21.15.138:0/3346953986 >> 172.21.15.138:0/3346953986 conn(0x5581a250ec00 msgr2=0x5581a2617000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- 172.21.15.138:0/3346953986 shutdown_connections 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- 172.21.15.138:0/3346953986 wait complete. 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.425+0000 7f144c536880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a254e8c0 con 0x5581a250f000 2024-04-27T05:21:33.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1448e85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1448e85640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42346/0 (socket says 172.21.15.138:42346) 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1448e85640 1 -- 172.21.15.138:0/1399777489 learned_addr learned my addr 172.21.15.138:0/1399777489 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1448e85640 1 -- 172.21.15.138:0/1399777489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a25136c0 con 0x5581a250f000 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1448e85640 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5581a33cade0 tx=0x5581a2620c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1445e7f640 1 -- 172.21.15.138:0/1399777489 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a2527e40 con 0x5581a250f000 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1445e7f640 1 -- 172.21.15.138:0/1399777489 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a32ee000 con 0x5581a250f000 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581a25131e0 con 0x5581a250f000 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f1445e7f640 1 -- 172.21.15.138:0/1399777489 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a25274a0 con 0x5581a250f000 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.426+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581a2580ea0 con 0x5581a250f000 2024-04-27T05:21:33.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1445e7f640 1 -- 172.21.15.138:0/1399777489 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5581a2538b00 con 0x5581a250f000 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f144c536880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f144c536880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a2532280 con 0x5581a3306c00 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1445e7f640 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a3307c00 0x5581a27f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1445e7f640 1 -- 172.21.15.138:0/1399777489 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5581a2521a40 con 0x5581a250f000 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1449686640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1449686640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42354/0 (socket says 172.21.15.138:42354) 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1448e85640 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a3307c00 0x5581a27f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1449686640 1 -- 172.21.15.138:0/2241044468 learned_addr learned my addr 172.21.15.138:0/2241044468 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1449686640 1 -- 172.21.15.138:0/2241044468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a2580d00 con 0x5581a3306c00 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1448e85640 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a3307c00 0x5581a27f1080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5581a340a2a0 tx=0x5581a2620cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.427+0000 7f1449686640 1 --2- 172.21.15.138:0/2241044468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5581a33cb590 tx=0x5581a2621c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c176abefa64e644 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144367a640 1 -- 172.21.15.138:0/2241044468 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a2527e40 con 0x5581a3306c00 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144367a640 1 -- 172.21.15.138:0/2241044468 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a32ee600 con 0x5581a3306c00 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144367a640 1 -- 172.21.15.138:0/2241044468 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a25274a0 con 0x5581a3306c00 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 -- 172.21.15.138:0/2241044468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 msgr2=0x5581a27f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 --2- 172.21.15.138:0/2241044468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5581a33cb590 tx=0x5581a2621c20 comp rx=0 tx=0).stop 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 -- 172.21.15.138:0/2241044468 shutdown_connections 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 --2- 172.21.15.138:0/2241044468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 -- 172.21.15.138:0/2241044468 >> 172.21.15.138:0/2241044468 conn(0x5581a250f400 msgr2=0x5581a2619000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 -- 172.21.15.138:0/2241044468 shutdown_connections 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 -- 172.21.15.138:0/2241044468 wait complete. 2024-04-27T05:21:33.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.428+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f144c536880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f144c536880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a2533180 con 0x5581a3306c00 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1449686640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1449686640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42370/0 (socket says 172.21.15.138:42370) 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 learned_addr learned my addr 172.21.15.138:0/3689510741 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a2580b60 con 0x5581a3306c00 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1449686640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5581a33135c0 tx=0x5581a2621980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1441e77640 1 -- 172.21.15.138:0/3689510741 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a2527e40 con 0x5581a3306c00 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1441e77640 1 -- 172.21.15.138:0/3689510741 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a32ee780 con 0x5581a3306c00 2024-04-27T05:21:33.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581a25809c0 con 0x5581a3306c00 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f1441e77640 1 -- 172.21.15.138:0/3689510741 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a25274a0 con 0x5581a3306c00 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.429+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581a2580820 con 0x5581a3306c00 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f1441e77640 1 -- 172.21.15.138:0/3689510741 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5581a2539600 con 0x5581a3306c00 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f1441e77640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a288c000 0x5581a27f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f1441e77640 1 -- 172.21.15.138:0/3689510741 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5581a2521c00 con 0x5581a3306c00 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f1448684640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a288c000 0x5581a27f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5581a3307000 0x5581a27f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307400 con 0x5581a3307000 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.430+0000 7f1449686640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5581a3307000 0x5581a27f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f1448684640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a288c000 0x5581a27f1600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5581a346b260 tx=0x5581a2852120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f1449686640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5581a3307000 0x5581a27f1b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 realm 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307800 con 0x5581a3307000 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288d000 con 0x5581a3307000 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288d400 con 0x5581a3307000 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.431+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5581a288d800 0x5581a27f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288dc00 con 0x5581a288d800 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f1448e85640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5581a288d800 0x5581a27f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f1448e85640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5581a288d800 0x5581a27f2100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 20 searching for the correct realm 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.432+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288d000 con 0x5581a288d800 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448e85640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5581a288d000 0x5581a27f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288dc00 con 0x5581a288d000 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448684640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5581a288d000 0x5581a27f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448684640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5581a288d000 0x5581a27f2c00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448684640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5581a288dc00 0x5581a27f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.433+0000 7f1449686640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5581a288dc00 0x5581a27f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5581a288dc00 0x5581a27f3180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.434+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.435+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.435+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.435+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.435+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d000 2024-04-27T05:21:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.435+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.435+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.436+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.437+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.438+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.439+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d000 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d800 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288dc00 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.440+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a3307000 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288cc00 con 0x5581a288d000 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288d400 con 0x5581a3307000 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307400 con 0x5581a3307000 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.441+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307800 con 0x5581a288d800 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 20 zone default found 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 4 Realm: () 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 20 zonegroup default 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0000 con 0x5581a288dc00 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.442+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0400 con 0x5581a288d800 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0800 con 0x5581a288d800 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 20 started sync module instance, tier type = 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.443+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0c00 con 0x5581a288d000 2024-04-27T05:21:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.444+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.444+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94015265343744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1000 con 0x5581a288d000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.444+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1400 con 0x5581a3307000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.445+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94015265343744] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.445+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.445+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94015265344896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1800 con 0x5581a3307000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.445+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1c00 con 0x5581a288d800 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.446+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94015265344896] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.446+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.446+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94015265346048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0400 con 0x5581a288d800 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.446+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0800 con 0x5581a288d000 2024-04-27T05:21:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.447+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94015265346048] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.447+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.447+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94015265347200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1c00 con 0x5581a288d000 2024-04-27T05:21:33.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.447+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0000 con 0x5581a3307000 2024-04-27T05:21:33.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.448+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.448+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94015265348352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307800 con 0x5581a3307000 2024-04-27T05:21:33.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.448+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307400 con 0x5581a288dc00 2024-04-27T05:21:33.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.450+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.450+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94015265347200] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.450+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94015265349504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a288d400 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.450+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a346cc00 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.450+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94015265348352] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.451+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94015265349504] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.451+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.451+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94015265350656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a346c800 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.451+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0c00 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94015265350656] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94015265351808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1000 con 0x5581a288dc00 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=0 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=1 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=2 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=3 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=4 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=5 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.452+0000 7f144c536880 20 add_watcher() i=6 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.453+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94015265351808] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.453+0000 7f144c536880 20 add_watcher() i=7 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.453+0000 7f144c536880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.453+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0800 con 0x5581a3307000 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.453+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.454+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94015265352960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1400 con 0x5581a3307000 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.454+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94015265352960] v29'56 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f142fe53640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f144c536880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f144c536880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f144c536880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f144c536880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a2532780 con 0x5581a288cc00 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f1448684640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f1448684640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42374/0 (socket says 172.21.15.138:42374) 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.455+0000 7f1448684640 1 -- 172.21.15.138:0/371341813 learned_addr learned my addr 172.21.15.138:0/371341813 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f1448684640 1 -- 172.21.15.138:0/371341813 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a2580680 con 0x5581a288cc00 2024-04-27T05:21:33.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f1448684640 1 --2- 172.21.15.138:0/371341813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5581a286e960 tx=0x5581a2852180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878e9f7faadaaff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f142f652640 1 -- 172.21.15.138:0/371341813 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a28b2580 con 0x5581a288cc00 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f142f652640 1 -- 172.21.15.138:0/371341813 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a32ee000 con 0x5581a288cc00 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f142f652640 1 -- 172.21.15.138:0/371341813 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a28b26e0 con 0x5581a288cc00 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f144c536880 1 -- 172.21.15.138:0/371341813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 msgr2=0x5581a27f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f144c536880 1 --2- 172.21.15.138:0/371341813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5581a286e960 tx=0x5581a2852180 comp rx=0 tx=0).stop 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f144c536880 1 -- 172.21.15.138:0/371341813 shutdown_connections 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.456+0000 7f144c536880 1 --2- 172.21.15.138:0/371341813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 -- 172.21.15.138:0/371341813 >> 172.21.15.138:0/371341813 conn(0x5581a28b1800 msgr2=0x5581a2619800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 -- 172.21.15.138:0/371341813 shutdown_connections 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 -- 172.21.15.138:0/371341813 wait complete. 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 Processor -- start 2024-04-27T05:21:33.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 -- start start 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f144c536880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a2532500 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f1448684640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f1448684640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42378/0 (socket says 172.21.15.138:42378) 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.457+0000 7f1448684640 1 -- 172.21.15.138:0/4128049722 learned_addr learned my addr 172.21.15.138:0/4128049722 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f1448684640 1 -- 172.21.15.138:0/4128049722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a25804e0 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f1448684640 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5581a286f890 tx=0x5581a2852ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f142de4f640 1 -- 172.21.15.138:0/4128049722 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a28b2840 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f142de4f640 1 -- 172.21.15.138:0/4128049722 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581a32eea80 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f142de4f640 1 -- 172.21.15.138:0/4128049722 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581a28b2580 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581a2513380 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.458+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581a2512ea0 con 0x5581a288cc00 2024-04-27T05:21:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f142de4f640 1 -- 172.21.15.138:0/4128049722 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5581a2539b80 con 0x5581a288cc00 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f142de4f640 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a346cc00 0x5581a27f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f142de4f640 1 -- 172.21.15.138:0/4128049722 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5581a2521dc0 con 0x5581a288cc00 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f144c536880 5 note: GC not initialized 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f1449686640 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a346cc00 0x5581a27f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f142b64a640 20 reqs_thread_entry: start 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f144c536880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f1449686640 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a346cc00 0x5581a27f3700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5581a286f260 tx=0x5581a28528a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f144c536880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f1429646640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:03 2024 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f1428e24640 20 reqs_thread_entry: start 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f144c536880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:33.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.459+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982000 con 0x5581a3307000 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982400 con 0x5581a3307000 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 197+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-04-27T05:21:33.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982800 con 0x5581a288dc00 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.460+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982c00 con 0x5581a3307000 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.461+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~357] v29'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.461+0000 7f144c536880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.461+0000 7f144c536880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.461+0000 7f144c536880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.461+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94015265355264 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2983000 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== watch-notify(notify (1) cookie 94015265348352 notify 124554051584 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5581a2851d40 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f144367a640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051584 cookie 94015265348352 notifier 4301 bl.length()=533 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(66 notify.4 [notify cookie 94015265355264 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f144367a640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f144367a640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f144367a640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0c00 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== watch-notify(notify_complete (2) cookie 94015265355264 notify 124554051584 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5581a290ba00 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.462+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2983400 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.463+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 239+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.463+0000 7f144c536880 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.463+0000 7f144c536880 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.463+0000 7f144c536880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-27T05:21:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.463+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94015265355264 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2983800 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== watch-notify(notify (1) cookie 94015265347200 notify 124554051584 ret 0) v3 ==== 211+0+0 (crc 0 0 0) 0x5581a290bd40 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f144367a640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051584 cookie 94015265347200 notifier 4301 bl.length()=169 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f144367a640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f144367a640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f144367a640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1000 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(69 notify.3 [notify cookie 94015265355264 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== watch-notify(notify_complete (2) cookie 94015265355264 notify 124554051584 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5581a2907860 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.464+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2983c00 con 0x5581a288dc00 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f144c536880 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f144c536880 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f144c536880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94015265355264 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982000 con 0x5581a288d000 2024-04-27T05:21:33.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== watch-notify(notify (1) cookie 94015265347200 notify 124554051585 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x5581a2907860 con 0x5581a288d000 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1442e79640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051585 cookie 94015265347200 notifier 4301 bl.length()=177 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1442e79640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1442e79640 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(72 notify.3 [notify cookie 94015265355264 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.465+0000 7f1442e79640 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2994000 con 0x5581a288d000 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.466+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.466+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== watch-notify(notify_complete (2) cookie 94015265355264 notify 124554051585 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5581a2904ea0 con 0x5581a288d000 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:33.492 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:33.493 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.494 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:33.495 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.466+0000 7f142fe53640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.466+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94015265352960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982400 con 0x5581a3307000 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.467+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94015265352960] v29'57 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.467+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94015265343744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982800 con 0x5581a288d000 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.468+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94015265343744] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.468+0000 7f144c536880 20 remove_watcher() i=0 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.468+0000 7f144c536880 2 removed watcher, disabling cache 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.468+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94015265344896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982c00 con 0x5581a3307000 2024-04-27T05:21:33.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.469+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94015265344896] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.469+0000 7f144c536880 20 remove_watcher() i=1 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.469+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94015265346048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2983000 con 0x5581a288d800 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.470+0000 7f1448e85640 1 -- 172.21.15.138:0/3689510741 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94015265346048] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6fc0 con 0x5581a288d800 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.470+0000 7f144c536880 20 remove_watcher() i=2 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.470+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94015265347200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2983800 con 0x5581a288d000 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.471+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94015265347200] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a7200 con 0x5581a288d000 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.471+0000 7f144c536880 20 remove_watcher() i=3 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.471+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94015265348352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a2982000 con 0x5581a3307000 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.472+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94015265348352] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a3307000 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.472+0000 7f144c536880 20 remove_watcher() i=4 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.472+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94015265349504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b0000 con 0x5581a288dc00 2024-04-27T05:21:33.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.473+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94015265349504] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.473+0000 7f144c536880 20 remove_watcher() i=5 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.473+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94015265350656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a28b1000 con 0x5581a288dc00 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.474+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94015265350656] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.474+0000 7f144c536880 20 remove_watcher() i=6 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.474+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94015265351808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581a3307400 con 0x5581a288dc00 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.474+0000 7f1449686640 1 -- 172.21.15.138:0/3689510741 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94015265351808] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5581a28a6900 con 0x5581a288dc00 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 20 remove_watcher() i=7 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5581a288dc00 msgr2=0x5581a27f3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5581a288dc00 0x5581a27f3180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5581a3307000 msgr2=0x5581a27f1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5581a3307000 0x5581a27f1b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5581a288d000 msgr2=0x5581a27f2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5581a288d000 0x5581a27f2c00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5581a288d800 msgr2=0x5581a27f2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5581a288d800 0x5581a27f2100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a288c000 msgr2=0x5581a27f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a288c000 0x5581a27f1600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5581a346b260 tx=0x5581a2852120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 msgr2=0x5581a27f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/3689510741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a3306c00 0x5581a27f0b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5581a33135c0 tx=0x5581a2621980 comp rx=0 tx=0).stop 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 reap_dead start 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f1448684640 1 -- 172.21.15.138:0/3689510741 reap_dead start 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 shutdown_connections 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 >> 172.21.15.138:0/3689510741 conn(0x5581a250f400 msgr2=0x5581a2619000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 shutdown_connections 2024-04-27T05:21:33.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/3689510741 wait complete. 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a346cc00 msgr2=0x5581a27f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a346cc00 0x5581a27f3700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5581a286f260 tx=0x5581a28528a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 msgr2=0x5581a27f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5581a286f890 tx=0x5581a2852ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 shutdown_connections 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a346cc00 0x5581a27f3700 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 --2- 172.21.15.138:0/4128049722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a288cc00 0x5581a27f2680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 >> 172.21.15.138:0/4128049722 conn(0x5581a28b1800 msgr2=0x5581a2619800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 shutdown_connections 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.475+0000 7f144c536880 1 -- 172.21.15.138:0/4128049722 wait complete. 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a3307c00 msgr2=0x5581a27f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a3307c00 0x5581a27f1080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5581a340a2a0 tx=0x5581a2620cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 msgr2=0x5581a27f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5581a33cade0 tx=0x5581a2620c60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 shutdown_connections 2024-04-27T05:21:33.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581a3307c00 0x5581a27f1080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 --2- 172.21.15.138:0/1399777489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5581a250f000 0x5581a27f0580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 >> 172.21.15.138:0/1399777489 conn(0x5581a250ec00 msgr2=0x5581a2617000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 shutdown_connections 2024-04-27T05:21:33.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.476+0000 7f144c536880 1 -- 172.21.15.138:0/1399777489 wait complete. 2024-04-27T05:21:33.507 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:33.507 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-27T05:21:33.507 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-27T05:21:33.508 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-04-27T05:21:33.553 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.553 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a5894880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5585028db000 0x558502bbc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a5894880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558502bb2600 con 0x5585028db400 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a5894880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55850291a280 con 0x5585028db000 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a4039640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5585028db000 0x558502bbc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a3838640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5585028db400 0x5585029e2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60720/0 (socket says 172.21.15.138:60720) 2024-04-27T05:21:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a3838640 1 -- 172.21.15.138:0/3250396548 learned_addr learned my addr 172.21.15.138:0/3250396548 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a4039640 1 -- 172.21.15.138:0/3250396548 >> v1:172.21.15.114:6789/0 conn(0x5585028db400 legacy=0x5585029e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.552+0000 7f94a4039640 1 -- 172.21.15.138:0/3250396548 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585028df380 con 0x5585028db000 2024-04-27T05:21:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.554+0000 7f94a4039640 1 --2- 172.21.15.138:0/3250396548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5585029e6d50 tx=0x5585029ecde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ebd6de3b215586 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.554+0000 7f94a3037640 1 -- 172.21.15.138:0/3250396548 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f3ce0 con 0x5585028db000 2024-04-27T05:21:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.554+0000 7f94a3037640 1 -- 172.21.15.138:0/3250396548 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558502bb3b00 con 0x5585028db000 2024-04-27T05:21:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.554+0000 7f94a3037640 1 -- 172.21.15.138:0/3250396548 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f3e40 con 0x5585028db000 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 -- 172.21.15.138:0/3250396548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 msgr2=0x558502bbc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 --2- 172.21.15.138:0/3250396548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x5585029e6d50 tx=0x5585029ecde0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 -- 172.21.15.138:0/3250396548 shutdown_connections 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 --2- 172.21.15.138:0/3250396548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 -- 172.21.15.138:0/3250396548 >> 172.21.15.138:0/3250396548 conn(0x5585028dac00 msgr2=0x5585029e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 -- 172.21.15.138:0/3250396548 shutdown_connections 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.555+0000 7f94a5894880 1 -- 172.21.15.138:0/3250396548 wait complete. 2024-04-27T05:21:33.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a5894880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a5894880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55850291a780 con 0x5585028db000 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a4039640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a4039640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42386/0 (socket says 172.21.15.138:42386) 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a4039640 1 -- 172.21.15.138:0/3535505023 learned_addr learned my addr 172.21.15.138:0/3535505023 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a4039640 1 -- 172.21.15.138:0/3535505023 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585028deea0 con 0x5585028db000 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.556+0000 7f94a4039640 1 --2- 172.21.15.138:0/3535505023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5585036df560 tx=0x5585029eccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7162f9306c5669b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a2836640 1 -- 172.21.15.138:0/3535505023 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f34a0 con 0x5585028db000 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a2836640 1 -- 172.21.15.138:0/3535505023 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558502bb3e00 con 0x5585028db000 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a2836640 1 -- 172.21.15.138:0/3535505023 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f3ce0 con 0x5585028db000 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- 172.21.15.138:0/3535505023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 msgr2=0x558502bbc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 --2- 172.21.15.138:0/3535505023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5585036df560 tx=0x5585029eccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- 172.21.15.138:0/3535505023 shutdown_connections 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 --2- 172.21.15.138:0/3535505023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- 172.21.15.138:0/3535505023 >> 172.21.15.138:0/3535505023 conn(0x5585028dac00 msgr2=0x5585029e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- 172.21.15.138:0/3535505023 shutdown_connections 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- 172.21.15.138:0/3535505023 wait complete. 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.557+0000 7f94a5894880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55850291a8c0 con 0x5585028db000 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a4039640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a4039640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42390/0 (socket says 172.21.15.138:42390) 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a4039640 1 -- 172.21.15.138:0/496470609 learned_addr learned my addr 172.21.15.138:0/496470609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a4039640 1 -- 172.21.15.138:0/496470609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585028df6c0 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a4039640 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x558503796de0 tx=0x5585029ecc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a1033640 1 -- 172.21.15.138:0/496470609 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f3e40 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a1033640 1 -- 172.21.15.138:0/496470609 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585036ba000 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585028df1e0 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a1033640 1 -- 172.21.15.138:0/496470609 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f34a0 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.558+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55850294cea0 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a1033640 1 -- 172.21.15.138:0/496470609 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558502904b00 con 0x5585028db000 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a5894880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a5894880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585028fe280 con 0x5585036d2c00 2024-04-27T05:21:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a1033640 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5585036d3c00 0x558502bbd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a1033640 1 -- 172.21.15.138:0/496470609 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5585028eda40 con 0x5585028db000 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a483a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a483a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42392/0 (socket says 172.21.15.138:42392) 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a483a640 1 -- 172.21.15.138:0/3593516385 learned_addr learned my addr 172.21.15.138:0/3593516385 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a4039640 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5585036d3c00 0x558502bbd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.559+0000 7f94a483a640 1 -- 172.21.15.138:0/3593516385 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850294cd00 con 0x5585036d2c00 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a4039640 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5585036d3c00 0x558502bbd080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5585037d62a0 tx=0x5585029eccc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a483a640 1 --2- 172.21.15.138:0/3593516385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x558503797a70 tx=0x5585029edc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef26e00405e2949a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f949e82e640 1 -- 172.21.15.138:0/3593516385 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f3e40 con 0x5585036d2c00 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f949e82e640 1 -- 172.21.15.138:0/3593516385 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585036ba600 con 0x5585036d2c00 2024-04-27T05:21:33.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f949e82e640 1 -- 172.21.15.138:0/3593516385 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f34a0 con 0x5585036d2c00 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 -- 172.21.15.138:0/3593516385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 msgr2=0x558502bbcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 --2- 172.21.15.138:0/3593516385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x558503797a70 tx=0x5585029edc20 comp rx=0 tx=0).stop 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 -- 172.21.15.138:0/3593516385 shutdown_connections 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 --2- 172.21.15.138:0/3593516385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 -- 172.21.15.138:0/3593516385 >> 172.21.15.138:0/3593516385 conn(0x5585028db400 msgr2=0x5585029e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 -- 172.21.15.138:0/3593516385 shutdown_connections 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 -- 172.21.15.138:0/3593516385 wait complete. 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.560+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a5894880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a5894880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585028ff180 con 0x5585036d2c00 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a483a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a483a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42408/0 (socket says 172.21.15.138:42408) 2024-04-27T05:21:33.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 learned_addr learned my addr 172.21.15.138:0/1313870900 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850294cb60 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a483a640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5585036df5c0 tx=0x5585029ed1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f949d02b640 1 -- 172.21.15.138:0/1313870900 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f3e40 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f949d02b640 1 -- 172.21.15.138:0/1313870900 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585036ba780 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55850294c9c0 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f949d02b640 1 -- 172.21.15.138:0/1313870900 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5585028f34a0 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.561+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55850294c820 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f949d02b640 1 -- 172.21.15.138:0/1313870900 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558502905600 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f949d02b640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558502c58000 0x558502bbd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f949d02b640 1 -- 172.21.15.138:0/1313870900 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5585028edc00 con 0x5585036d2c00 2024-04-27T05:21:33.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f94a3838640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558502c58000 0x558502bbd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5585036d3000 0x558502bbdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5585036d3400 con 0x5585036d3000 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.562+0000 7f94a483a640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5585036d3000 0x558502bbdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a3838640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558502c58000 0x558502bbd600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55850383b260 tx=0x558502c20120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a483a640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5585036d3000 0x558502bbdb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 realm 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5585036d3800 con 0x5585036d3000 2024-04-27T05:21:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59000 con 0x5585036d3000 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.563+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59400 con 0x5585036d3000 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558502c59800 0x558502bbe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59c00 con 0x558502c59800 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a4039640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558502c59800 0x558502bbe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a4039640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558502c59800 0x558502bbe100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 20 searching for the correct realm 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.564+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59000 con 0x558502c59800 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a4039640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558502c59000 0x558502bbec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59c00 con 0x558502c59000 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a3838640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558502c59000 0x558502bbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a3838640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558502c59000 0x558502bbec00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a3838640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558502c59c00 0x558502bbf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.565+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.566+0000 7f94a483a640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558502c59c00 0x558502bbf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.566+0000 7f94a483a640 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558502c59c00 0x558502bbf180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.566+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.566+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.566+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.566+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59000 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.567+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59000 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.568+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.569+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.570+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59000 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.571+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.572+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.572+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59000 2024-04-27T05:21:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.572+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.572+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59800 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.572+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.572+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59c00 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x5585036d3000 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c58c00 con 0x558502c59000 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59400 con 0x5585036d3000 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5585036d3400 con 0x5585036d3000 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.573+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5585036d3800 con 0x558502c59800 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 20 zone default found 2024-04-27T05:21:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 4 Realm: () 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 20 zonegroup default 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c000 con 0x558502c59c00 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.574+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c400 con 0x558502c59800 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c800 con 0x558502c59800 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 20 started sync module instance, tier type = 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.575+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7cc00 con 0x558502c59000 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.576+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.576+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94029764839680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7d000 con 0x558502c59000 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.576+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7d400 con 0x5585036d3000 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.577+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94029764839680] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.577+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.577+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94029764840832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7d800 con 0x5585036d3000 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.577+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7dc00 con 0x558502c59800 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.578+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94029764840832] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.578+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.578+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94029764841984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c400 con 0x558502c59800 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.578+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c800 con 0x558502c59000 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.579+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.579+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94029764841984] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.579+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94029764843136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7dc00 con 0x558502c59000 2024-04-27T05:21:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.579+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c000 con 0x5585036d3000 2024-04-27T05:21:33.615 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.580+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.580+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94029764844288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5585036d3800 con 0x5585036d3000 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.580+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5585036d3400 con 0x558502c59c00 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.580+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94029764843136] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.581+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94029764844288] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.583+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.583+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94029764845440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c59400 con 0x558502c59c00 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.583+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55850383d800 con 0x558502c59c00 2024-04-27T05:21:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.584+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94029764845440] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.584+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.584+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94029764846592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55850383d400 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.584+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7cc00 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94029764846592] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94029764847744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7d000 con 0x558502c59c00 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=0 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=1 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=2 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=3 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=4 2024-04-27T05:21:33.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=5 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.585+0000 7f94a5894880 20 add_watcher() i=6 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.586+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94029764847744] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.586+0000 7f94a5894880 20 add_watcher() i=7 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.586+0000 7f94a5894880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.586+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7c800 con 0x5585036d3000 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.586+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.586+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94029764848896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502c7d400 con 0x5585036d3000 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.587+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94029764848896] v29'58 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.587+0000 7f948b007640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a5894880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a5894880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a5894880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a5894880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585028fe780 con 0x558502c58c00 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a3838640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a3838640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42416/0 (socket says 172.21.15.138:42416) 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a3838640 1 -- 172.21.15.138:0/2541646439 learned_addr learned my addr 172.21.15.138:0/2541646439 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a3838640 1 -- 172.21.15.138:0/2541646439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850294c680 con 0x558502c58c00 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.588+0000 7f94a3838640 1 --2- 172.21.15.138:0/2541646439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x558502c3a930 tx=0x558502c20180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4660c5a1dd50a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f948a806640 1 -- 172.21.15.138:0/2541646439 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558502c7e580 con 0x558502c58c00 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f948a806640 1 -- 172.21.15.138:0/2541646439 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585036ba000 con 0x558502c58c00 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f948a806640 1 -- 172.21.15.138:0/2541646439 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558502c7e6e0 con 0x558502c58c00 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 -- 172.21.15.138:0/2541646439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 msgr2=0x558502bbe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 --2- 172.21.15.138:0/2541646439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x558502c3a930 tx=0x558502c20180 comp rx=0 tx=0).stop 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 -- 172.21.15.138:0/2541646439 shutdown_connections 2024-04-27T05:21:33.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 --2- 172.21.15.138:0/2541646439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 -- 172.21.15.138:0/2541646439 >> 172.21.15.138:0/2541646439 conn(0x558502c7d800 msgr2=0x5585029e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 -- 172.21.15.138:0/2541646439 shutdown_connections 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.589+0000 7f94a5894880 1 -- 172.21.15.138:0/2541646439 wait complete. 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a5894880 1 Processor -- start 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a5894880 1 -- start start 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a5894880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a5894880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585028fe500 con 0x558502c58c00 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a3838640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a3838640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42418/0 (socket says 172.21.15.138:42418) 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a3838640 1 -- 172.21.15.138:0/1912120836 learned_addr learned my addr 172.21.15.138:0/1912120836 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a3838640 1 -- 172.21.15.138:0/1912120836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55850294c4e0 con 0x558502c58c00 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a3838640 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x558502c3b860 tx=0x558502c20ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f9489003640 1 -- 172.21.15.138:0/1912120836 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558502c7e840 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f9489003640 1 -- 172.21.15.138:0/1912120836 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5585036baa80 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f9489003640 1 -- 172.21.15.138:0/1912120836 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558502c7e580 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.590+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585028df380 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.591+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585028deea0 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.591+0000 7f9489003640 1 -- 172.21.15.138:0/1912120836 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558502905b80 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.591+0000 7f9489003640 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55850383d800 0x558502bbf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.591+0000 7f9489003640 1 -- 172.21.15.138:0/1912120836 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5585028eddc0 con 0x558502c58c00 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.591+0000 7f94a5894880 5 note: GC not initialized 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.591+0000 7f94a483a640 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55850383d800 0x558502bbf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94867fe640 20 reqs_thread_entry: start 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a483a640 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55850383d800 0x558502bbf700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x558502c3b230 tx=0x558502c208a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94847fa640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:03 2024 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f9483fd8640 20 reqs_thread_entry: start 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4e000 con 0x558502c59c00 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:33.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.592+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4e400 con 0x5585036d3000 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+23 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 20 rados_obj.operate() r=0 bl.length=7 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4e800 con 0x5585036d3000 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 foo [call out=48b,read 0~357 out=357b] v0'0 uv1 ondisk = 0) v8 ==== 189+0+405 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:33.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x11 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.593+0000 7f94a5894880 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.594+0000 7f948b007640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.594+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94029764848896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4ec00 con 0x5585036d3000 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.595+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94029764848896] v29'59 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.595+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94029764839680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4f000 con 0x558502c59000 2024-04-27T05:21:33.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.596+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94029764839680] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.596+0000 7f94a5894880 20 remove_watcher() i=0 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.596+0000 7f94a5894880 2 removed watcher, disabling cache 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.596+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94029764840832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4f400 con 0x5585036d3000 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.597+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94029764840832] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.597+0000 7f94a5894880 20 remove_watcher() i=1 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.597+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94029764841984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4f800 con 0x558502c59800 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.598+0000 7f94a4039640 1 -- 172.21.15.138:0/1313870900 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94029764841984] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72d80 con 0x558502c59800 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.598+0000 7f94a5894880 20 remove_watcher() i=2 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.598+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94029764843136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4fc00 con 0x558502c59000 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.599+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94029764843136] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c72fc0 con 0x558502c59000 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.599+0000 7f94a5894880 20 remove_watcher() i=3 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.599+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94029764844288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4e000 con 0x5585036d3000 2024-04-27T05:21:33.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.599+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94029764844288] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x5585036d3000 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.599+0000 7f94a5894880 20 remove_watcher() i=4 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.599+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94029764845440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4e400 con 0x558502c59c00 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.600+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94029764845440] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.600+0000 7f94a5894880 20 remove_watcher() i=5 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.600+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94029764846592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4e800 con 0x558502c59c00 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.601+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94029764846592] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.601+0000 7f94a5894880 20 remove_watcher() i=6 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.601+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94029764847744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558502d4ec00 con 0x558502c59c00 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a483a640 1 -- 172.21.15.138:0/1313870900 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94029764847744] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558502c726c0 con 0x558502c59c00 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 20 remove_watcher() i=7 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558502c59c00 msgr2=0x558502bbf180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558502c59c00 0x558502bbf180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5585036d3000 msgr2=0x558502bbdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5585036d3000 0x558502bbdb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558502c59000 msgr2=0x558502bbec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558502c59000 0x558502bbec00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558502c59800 msgr2=0x558502bbe100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558502c59800 0x558502bbe100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558502c58000 msgr2=0x558502bbd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558502c58000 0x558502bbd600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55850383b260 tx=0x558502c20120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 msgr2=0x558502bbcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 --2- 172.21.15.138:0/1313870900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585036d2c00 0x558502bbcb00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5585036df5c0 tx=0x5585029ed1a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 reap_dead start 2024-04-27T05:21:33.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a3838640 1 -- 172.21.15.138:0/1313870900 reap_dead start 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 shutdown_connections 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 >> 172.21.15.138:0/1313870900 conn(0x5585028db400 msgr2=0x5585029e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 shutdown_connections 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.602+0000 7f94a5894880 1 -- 172.21.15.138:0/1313870900 wait complete. 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55850383d800 msgr2=0x558502bbf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55850383d800 0x558502bbf700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x558502c3b230 tx=0x558502c208a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 msgr2=0x558502bbe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x558502c3b860 tx=0x558502c20ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 shutdown_connections 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55850383d800 0x558502bbf700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/1912120836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558502c58c00 0x558502bbe680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 >> 172.21.15.138:0/1912120836 conn(0x558502c7d800 msgr2=0x5585029e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 shutdown_connections 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/1912120836 wait complete. 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5585036d3c00 msgr2=0x558502bbd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5585036d3c00 0x558502bbd080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5585037d62a0 tx=0x5585029eccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 msgr2=0x558502bbc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x558503796de0 tx=0x5585029ecc60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 shutdown_connections 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5585036d3c00 0x558502bbd080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 --2- 172.21.15.138:0/496470609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5585028db000 0x558502bbc580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 >> 172.21.15.138:0/496470609 conn(0x5585028dac00 msgr2=0x5585029e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 shutdown_connections 2024-04-27T05:21:33.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.603+0000 7f94a5894880 1 -- 172.21.15.138:0/496470609 wait complete. 2024-04-27T05:21:33.631 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:33.631 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:33.631 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c680e880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ca0c6b5400 0x55ca0c996580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c680e880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ca0c98c600 con 0x55ca0c6b5000 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c680e880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ca0c6f4280 con 0x55ca0c6b5400 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c47b2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ca0c6b5400 0x55ca0c996580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c47b2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ca0c6b5400 0x55ca0c996580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42430/0 (socket says 172.21.15.138:42430) 2024-04-27T05:21:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c47b2640 1 -- 172.21.15.138:0/2949726501 learned_addr learned my addr 172.21.15.138:0/2949726501 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c4fb3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55ca0c6b5000 0x55ca0c7bc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60732/0 (socket says 172.21.15.138:60732) 2024-04-27T05:21:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c47b2640 1 -- 172.21.15.138:0/2949726501 >> v1:172.21.15.114:6789/0 conn(0x55ca0c6b5000 legacy=0x55ca0c7bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.672+0000 7f09c47b2640 1 -- 172.21.15.138:0/2949726501 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c6b9380 con 0x55ca0c6b5400 2024-04-27T05:21:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c47b2640 1 --2- 172.21.15.138:0/2949726501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5400 0x55ca0c996580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55ca0c7c0c00 tx=0x55ca0c7c6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998905e47fa20a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c3fb1640 1 -- 172.21.15.138:0/2949726501 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cdce0 con 0x55ca0c6b5400 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c3fb1640 1 -- 172.21.15.138:0/2949726501 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0c98db00 con 0x55ca0c6b5400 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c3fb1640 1 -- 172.21.15.138:0/2949726501 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cde40 con 0x55ca0c6b5400 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 -- 172.21.15.138:0/2949726501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5400 msgr2=0x55ca0c996580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 --2- 172.21.15.138:0/2949726501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5400 0x55ca0c996580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55ca0c7c0c00 tx=0x55ca0c7c6de0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 -- 172.21.15.138:0/2949726501 shutdown_connections 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 --2- 172.21.15.138:0/2949726501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5400 0x55ca0c996580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 -- 172.21.15.138:0/2949726501 >> 172.21.15.138:0/2949726501 conn(0x55ca0c6b4c00 msgr2=0x55ca0c7bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 -- 172.21.15.138:0/2949726501 shutdown_connections 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.674+0000 7f09c680e880 1 -- 172.21.15.138:0/2949726501 wait complete. 2024-04-27T05:21:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.675+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.675+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.675+0000 7f09c680e880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.675+0000 7f09c680e880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c6f4780 con 0x55ca0c6b5000 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c4fb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c4fb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42444/0 (socket says 172.21.15.138:42444) 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c4fb3640 1 -- 172.21.15.138:0/2427709700 learned_addr learned my addr 172.21.15.138:0/2427709700 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c4fb3640 1 -- 172.21.15.138:0/2427709700 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c6b8ea0 con 0x55ca0c6b5000 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/2427709700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ca0d571290 tx=0x55ca0c7c6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bab7cc24069f5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c37b0640 1 -- 172.21.15.138:0/2427709700 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cd4a0 con 0x55ca0c6b5000 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c37b0640 1 -- 172.21.15.138:0/2427709700 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0c98de00 con 0x55ca0c6b5000 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c37b0640 1 -- 172.21.15.138:0/2427709700 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cdce0 con 0x55ca0c6b5000 2024-04-27T05:21:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c680e880 1 -- 172.21.15.138:0/2427709700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 msgr2=0x55ca0c996580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.676+0000 7f09c680e880 1 --2- 172.21.15.138:0/2427709700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55ca0d571290 tx=0x55ca0c7c6cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 -- 172.21.15.138:0/2427709700 shutdown_connections 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 --2- 172.21.15.138:0/2427709700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 -- 172.21.15.138:0/2427709700 >> 172.21.15.138:0/2427709700 conn(0x55ca0c6b4c00 msgr2=0x55ca0c7bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 -- 172.21.15.138:0/2427709700 shutdown_connections 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 -- 172.21.15.138:0/2427709700 wait complete. 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c680e880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c6f48c0 con 0x55ca0c6b5000 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c4fb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c4fb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42446/0 (socket says 172.21.15.138:42446) 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1713029446 learned_addr learned my addr 172.21.15.138:0/1713029446 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.677+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1713029446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c6b96c0 con 0x55ca0c6b5000 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55ca0d570b40 tx=0x55ca0c7c6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c1fad640 1 -- 172.21.15.138:0/1713029446 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cde40 con 0x55ca0c6b5000 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c1fad640 1 -- 172.21.15.138:0/1713029446 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0d586000 con 0x55ca0c6b5000 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c1fad640 1 -- 172.21.15.138:0/1713029446 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cd4a0 con 0x55ca0c6b5000 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca0c6b91e0 con 0x55ca0c6b5000 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca0c726ea0 con 0x55ca0c6b5000 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c680e880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c680e880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c6f4c80 con 0x55ca0d4ad800 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.678+0000 7f09c1fad640 1 -- 172.21.15.138:0/1713029446 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca0c6df600 con 0x55ca0c6b5000 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c1fad640 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d60a000 0x55ca0c997080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c57b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c1fad640 1 -- 172.21.15.138:0/1713029446 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55ca0c6c7a40 con 0x55ca0c6b5000 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c57b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42448/0 (socket says 172.21.15.138:42448) 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c57b4640 1 -- 172.21.15.138:0/4071488909 learned_addr learned my addr 172.21.15.138:0/4071488909 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d60a000 0x55ca0c997080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c57b4640 1 -- 172.21.15.138:0/4071488909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c726d00 con 0x55ca0d4ad800 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d60a000 0x55ca0c997080 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55ca0d5b6a50 tx=0x55ca0c7c74a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09c57b4640 1 --2- 172.21.15.138:0/4071488909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55ca0d571e90 tx=0x55ca0c7c6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8150a693a6c6d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09bf7a8640 1 -- 172.21.15.138:0/4071488909 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cde40 con 0x55ca0d4ad800 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09bf7a8640 1 -- 172.21.15.138:0/4071488909 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0d586300 con 0x55ca0d4ad800 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.679+0000 7f09bf7a8640 1 -- 172.21.15.138:0/4071488909 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cd4a0 con 0x55ca0d4ad800 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- 172.21.15.138:0/4071488909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 msgr2=0x55ca0c996b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 --2- 172.21.15.138:0/4071488909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55ca0d571e90 tx=0x55ca0c7c6c00 comp rx=0 tx=0).stop 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- 172.21.15.138:0/4071488909 shutdown_connections 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 --2- 172.21.15.138:0/4071488909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- 172.21.15.138:0/4071488909 >> 172.21.15.138:0/4071488909 conn(0x55ca0c6b5400 msgr2=0x55ca0c7bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- 172.21.15.138:0/4071488909 shutdown_connections 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- 172.21.15.138:0/4071488909 wait complete. 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c680e880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c6d8280 con 0x55ca0d4ad800 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c57b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c57b4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42460/0 (socket says 172.21.15.138:42460) 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 learned_addr learned my addr 172.21.15.138:0/1260123596 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.680+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c726b60 con 0x55ca0d4ad800 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55ca0d5719e0 tx=0x55ca0c7c71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09bdfa5640 1 -- 172.21.15.138:0/1260123596 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cde40 con 0x55ca0d4ad800 2024-04-27T05:21:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09bdfa5640 1 -- 172.21.15.138:0/1260123596 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0d586480 con 0x55ca0d4ad800 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca0c7269c0 con 0x55ca0d4ad800 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09bdfa5640 1 -- 172.21.15.138:0/1260123596 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0c6cd4a0 con 0x55ca0d4ad800 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca0c726820 con 0x55ca0d4ad800 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.681+0000 7f09bdfa5640 1 -- 172.21.15.138:0/1260123596 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca0c6df8c0 con 0x55ca0d4ad800 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09bdfa5640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0ca46000 0x55ca0c997600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09bdfa5640 1 -- 172.21.15.138:0/1260123596 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55ca0c6c7c00 con 0x55ca0d4ad800 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c47b2640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0ca46000 0x55ca0c997600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca0d4adc00 0x55ca0c997b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46400 con 0x55ca0d4adc00 2024-04-27T05:21:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca0d4adc00 0x55ca0c997b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c47b2640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0ca46000 0x55ca0c997600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55ca0c7c0c00 tx=0x55ca0ca0c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca0d4adc00 0x55ca0c997b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 20 realm 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.682+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46800 con 0x55ca0d4adc00 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46c00 con 0x55ca0d4adc00 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47000 con 0x55ca0d4adc00 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca0ca47400 0x55ca0c998100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46400 con 0x55ca0ca47400 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca0ca47400 0x55ca0c998100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.683+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca0ca47400 0x55ca0c998100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c680e880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c680e880 20 searching for the correct realm 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46800 con 0x55ca0ca47400 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c4fb3640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca0ca46800 0x55ca0c998c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46400 con 0x55ca0ca46800 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c47b2640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca0ca46800 0x55ca0c998c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.684+0000 7f09c47b2640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca0ca46800 0x55ca0c998c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c47b2640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca0ca46400 0x55ca0c999180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca0ca46400 0x55ca0c999180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca0ca46400 0x55ca0c999180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.685+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46800 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.686+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.687+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46800 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.688+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.689+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46800 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.690+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.691+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.691+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46800 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.691+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.691+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca47400 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.691+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.691+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46400 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0d4adc00 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d4ac800 con 0x55ca0ca46800 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46c00 con 0x55ca0d4adc00 2024-04-27T05:21:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.692+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47000 con 0x55ca0d4adc00 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47800 con 0x55ca0ca47400 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 zone default found 2024-04-27T05:21:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 4 Realm: () 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 zonegroup default 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47c00 con 0x55ca0ca46400 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498000 con 0x55ca0ca47400 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.693+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498400 con 0x55ca0ca47400 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.694+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.694+0000 7f09c680e880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.694+0000 7f09c680e880 20 started sync module instance, tier type = 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.694+0000 7f09c680e880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.694+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498800 con 0x55ca0ca46800 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.695+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.695+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94326283102464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498c00 con 0x55ca0ca46800 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.695+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d499000 con 0x55ca0d4adc00 2024-04-27T05:21:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.695+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94326283102464] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.696+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.696+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94326283103616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d499400 con 0x55ca0d4adc00 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.696+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d499800 con 0x55ca0ca47400 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.696+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94326283103616] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94326283104768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498000 con 0x55ca0ca47400 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498400 con 0x55ca0ca46800 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94326283105920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d499800 con 0x55ca0ca46800 2024-04-27T05:21:33.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94326283104768] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.697+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47800 con 0x55ca0d4adc00 2024-04-27T05:21:33.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.698+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94326283107072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca46c00 con 0x55ca0d4adc00 2024-04-27T05:21:33.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47000 con 0x55ca0ca46400 2024-04-27T05:21:33.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94326283105920] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94326283107072] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94326283108224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0ca47c00 con 0x55ca0ca46400 2024-04-27T05:21:33.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.699+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d60b800 con 0x55ca0ca46400 2024-04-27T05:21:33.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.700+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94326283108224] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.700+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.700+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94326283109376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498800 con 0x55ca0ca46400 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.700+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498c00 con 0x55ca0ca46400 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94326283109376] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94326283110528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d498400 con 0x55ca0ca46400 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=0 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=1 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=2 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=3 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=4 2024-04-27T05:21:33.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=5 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.701+0000 7f09c680e880 20 add_watcher() i=6 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.702+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94326283110528] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.702+0000 7f09c680e880 20 add_watcher() i=7 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.702+0000 7f09c680e880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.702+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d499800 con 0x55ca0d4adc00 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.703+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.703+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94326283111680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0d499000 con 0x55ca0d4adc00 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.703+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94326283111680] v29'60 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.703+0000 7f09abf81640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c680e880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c680e880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c680e880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c680e880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c6d88c0 con 0x55ca0d499c00 2024-04-27T05:21:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c47b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c47b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42464/0 (socket says 172.21.15.138:42464) 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c47b2640 1 -- 172.21.15.138:0/536727553 learned_addr learned my addr 172.21.15.138:0/536727553 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c47b2640 1 -- 172.21.15.138:0/536727553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c726680 con 0x55ca0d499c00 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.704+0000 7f09c47b2640 1 --2- 172.21.15.138:0/536727553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55ca0d632b10 tx=0x55ca0ca0c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ebcf44f53c8fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09ab780640 1 -- 172.21.15.138:0/536727553 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0d49a580 con 0x55ca0d499c00 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09ab780640 1 -- 172.21.15.138:0/536727553 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0d586000 con 0x55ca0d499c00 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09ab780640 1 -- 172.21.15.138:0/536727553 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0d49a6e0 con 0x55ca0d499c00 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 -- 172.21.15.138:0/536727553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 msgr2=0x55ca0c998680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 --2- 172.21.15.138:0/536727553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55ca0d632b10 tx=0x55ca0ca0c0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 -- 172.21.15.138:0/536727553 shutdown_connections 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 --2- 172.21.15.138:0/536727553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 -- 172.21.15.138:0/536727553 >> 172.21.15.138:0/536727553 conn(0x55ca0d499400 msgr2=0x55ca0c7bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 -- 172.21.15.138:0/536727553 shutdown_connections 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 -- 172.21.15.138:0/536727553 wait complete. 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.705+0000 7f09c680e880 1 Processor -- start 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c680e880 1 -- start start 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c680e880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c680e880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca0c6d8780 con 0x55ca0d499c00 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c47b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c47b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42478/0 (socket says 172.21.15.138:42478) 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c47b2640 1 -- 172.21.15.138:0/1750347085 learned_addr learned my addr 172.21.15.138:0/1750347085 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c47b2640 1 -- 172.21.15.138:0/1750347085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca0c7264e0 con 0x55ca0d499c00 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c47b2640 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55ca0ca27620 tx=0x55ca0ca0c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09a9f7d640 1 -- 172.21.15.138:0/1750347085 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0d49a840 con 0x55ca0d499c00 2024-04-27T05:21:33.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09a9f7d640 1 -- 172.21.15.138:0/1750347085 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca0d586780 con 0x55ca0d499c00 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09a9f7d640 1 -- 172.21.15.138:0/1750347085 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca0d49a580 con 0x55ca0d499c00 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca0c6b8ea0 con 0x55ca0d499c00 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.706+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca0c6b96c0 con 0x55ca0d499c00 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.707+0000 7f09a9f7d640 1 -- 172.21.15.138:0/1750347085 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca0c6deb00 con 0x55ca0d499c00 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.707+0000 7f09a9f7d640 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d498800 0x55ca0c999700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.707+0000 7f09a9f7d640 1 -- 172.21.15.138:0/1750347085 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55ca0c6c7dc0 con 0x55ca0d499c00 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.707+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d498800 0x55ca0c999700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.707+0000 7f09c680e880 5 note: GC not initialized 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09a7778640 20 reqs_thread_entry: start 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c680e880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c57b4640 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d498800 0x55ca0c999700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55ca0ca26f30 tx=0x55ca0ca0c960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c680e880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:33.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09a5774640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09a4f52640 20 reqs_thread_entry: start 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c680e880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2a000 con 0x55ca0d4adc00 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.708+0000 7f09c680e880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:33.735 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:33.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.709+0000 7f09abf81640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.709+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94326283111680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2a400 con 0x55ca0d4adc00 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.710+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94326283111680] v29'61 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.710+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94326283102464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2a800 con 0x55ca0ca46800 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.711+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94326283102464] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.711+0000 7f09c680e880 20 remove_watcher() i=0 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.711+0000 7f09c680e880 2 removed watcher, disabling cache 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.711+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94326283103616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2ac00 con 0x55ca0d4adc00 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.712+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94326283103616] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.712+0000 7f09c680e880 20 remove_watcher() i=1 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.712+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94326283104768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2b000 con 0x55ca0ca47400 2024-04-27T05:21:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.713+0000 7f09c4fb3640 1 -- 172.21.15.138:0/1260123596 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94326283104768] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5afc0 con 0x55ca0ca47400 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.713+0000 7f09c680e880 20 remove_watcher() i=2 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.713+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94326283105920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2b400 con 0x55ca0ca46800 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.714+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94326283105920] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5b200 con 0x55ca0ca46800 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.714+0000 7f09c680e880 20 remove_watcher() i=3 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.714+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94326283107072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2b800 con 0x55ca0d4adc00 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.715+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94326283107072] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0d4adc00 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.715+0000 7f09c680e880 20 remove_watcher() i=4 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.715+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94326283108224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2bc00 con 0x55ca0ca46400 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.716+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94326283108224] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.716+0000 7f09c680e880 20 remove_watcher() i=5 2024-04-27T05:21:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.716+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94326283109376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2a000 con 0x55ca0ca46400 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94326283109376] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 20 remove_watcher() i=6 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94326283110528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ca0cb2a400 con 0x55ca0ca46400 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c57b4640 1 -- 172.21.15.138:0/1260123596 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94326283110528] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca0ca5a900 con 0x55ca0ca46400 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 20 remove_watcher() i=7 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca0ca46400 msgr2=0x55ca0c999180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca0ca46400 0x55ca0c999180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca0d4adc00 msgr2=0x55ca0c997b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca0d4adc00 0x55ca0c997b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca0ca46800 msgr2=0x55ca0c998c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca0ca46800 0x55ca0c998c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca0ca47400 msgr2=0x55ca0c998100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.717+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca0ca47400 0x55ca0c998100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0ca46000 msgr2=0x55ca0c997600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0ca46000 0x55ca0c997600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55ca0c7c0c00 tx=0x55ca0ca0c120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 msgr2=0x55ca0c996b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 --2- 172.21.15.138:0/1260123596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d4ad800 0x55ca0c996b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55ca0d5719e0 tx=0x55ca0c7c71a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 reap_dead start 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c47b2640 1 -- 172.21.15.138:0/1260123596 reap_dead start 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 shutdown_connections 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 >> 172.21.15.138:0/1260123596 conn(0x55ca0c6b5400 msgr2=0x55ca0c7bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 shutdown_connections 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1260123596 wait complete. 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d498800 msgr2=0x55ca0c999700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d498800 0x55ca0c999700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55ca0ca26f30 tx=0x55ca0ca0c960 comp rx=0 tx=0).stop 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 msgr2=0x55ca0c998680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55ca0ca27620 tx=0x55ca0ca0c000 comp rx=0 tx=0).stop 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 shutdown_connections 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d498800 0x55ca0c999700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 --2- 172.21.15.138:0/1750347085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0d499c00 0x55ca0c998680 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 >> 172.21.15.138:0/1750347085 conn(0x55ca0d499400 msgr2=0x55ca0c7bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 shutdown_connections 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.718+0000 7f09c680e880 1 -- 172.21.15.138:0/1750347085 wait complete. 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d60a000 msgr2=0x55ca0c997080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d60a000 0x55ca0c997080 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55ca0d5b6a50 tx=0x55ca0c7c74a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 msgr2=0x55ca0c996580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55ca0d570b40 tx=0x55ca0c7c6c60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 shutdown_connections 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca0d60a000 0x55ca0c997080 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 --2- 172.21.15.138:0/1713029446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca0c6b5000 0x55ca0c996580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 >> 172.21.15.138:0/1713029446 conn(0x55ca0c6b4c00 msgr2=0x55ca0c7bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 shutdown_connections 2024-04-27T05:21:33.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.719+0000 7f09c680e880 1 -- 172.21.15.138:0/1713029446 wait complete. 2024-04-27T05:21:33.750 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:33.750 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-27T05:21:33.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-27T05:21:33.751 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-27T05:21:33.827 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.827 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f333cb42880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f333cb42880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559bd5de2600 con 0x559bd5b0b400 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f333cb42880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x559bd5b4a280 con 0x559bd5b0b000 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f3339543640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f3339543640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42486/0 (socket says 172.21.15.138:42486) 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f3338d42640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x559bd5b0b400 0x559bd5c12800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60734/0 (socket says 172.21.15.138:60734) 2024-04-27T05:21:33.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f3339543640 1 -- 172.21.15.138:0/3544558246 learned_addr learned my addr 172.21.15.138:0/3544558246 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f3339543640 1 -- 172.21.15.138:0/3544558246 >> v1:172.21.15.114:6789/0 conn(0x559bd5b0b400 legacy=0x559bd5c12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.826+0000 7f3339543640 1 -- 172.21.15.138:0/3544558246 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b0f380 con 0x559bd5b0b000 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f3339543640 1 --2- 172.21.15.138:0/3544558246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x559bd5c16d50 tx=0x559bd5c1cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a718086d1944ef18 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f3338541640 1 -- 172.21.15.138:0/3544558246 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b23ce0 con 0x559bd5b0b000 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f3338541640 1 -- 172.21.15.138:0/3544558246 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd5de3b00 con 0x559bd5b0b000 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f3338541640 1 -- 172.21.15.138:0/3544558246 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b23e40 con 0x559bd5b0b000 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f333cb42880 1 -- 172.21.15.138:0/3544558246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 msgr2=0x559bd5dec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f333cb42880 1 --2- 172.21.15.138:0/3544558246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x559bd5c16d50 tx=0x559bd5c1cde0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f333cb42880 1 -- 172.21.15.138:0/3544558246 shutdown_connections 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f333cb42880 1 --2- 172.21.15.138:0/3544558246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.828+0000 7f333cb42880 1 -- 172.21.15.138:0/3544558246 >> 172.21.15.138:0/3544558246 conn(0x559bd5b0ac00 msgr2=0x559bd5c12000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.829+0000 7f333cb42880 1 -- 172.21.15.138:0/3544558246 shutdown_connections 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.829+0000 7f333cb42880 1 -- 172.21.15.138:0/3544558246 wait complete. 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.829+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.829+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.829+0000 7f333cb42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.829+0000 7f333cb42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b4a780 con 0x559bd5b0b000 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3339543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3339543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42494/0 (socket says 172.21.15.138:42494) 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3339543640 1 -- 172.21.15.138:0/2044057894 learned_addr learned my addr 172.21.15.138:0/2044057894 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3339543640 1 -- 172.21.15.138:0/2044057894 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b0eea0 con 0x559bd5b0b000 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3339543640 1 --2- 172.21.15.138:0/2044057894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x559bd690f560 tx=0x559bd5c1ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a87cce44952bde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3337d40640 1 -- 172.21.15.138:0/2044057894 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b234a0 con 0x559bd5b0b000 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3337d40640 1 -- 172.21.15.138:0/2044057894 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd5de3e00 con 0x559bd5b0b000 2024-04-27T05:21:33.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.830+0000 7f3337d40640 1 -- 172.21.15.138:0/2044057894 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b23ce0 con 0x559bd5b0b000 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- 172.21.15.138:0/2044057894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 msgr2=0x559bd5dec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 --2- 172.21.15.138:0/2044057894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x559bd690f560 tx=0x559bd5c1ccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- 172.21.15.138:0/2044057894 shutdown_connections 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 --2- 172.21.15.138:0/2044057894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- 172.21.15.138:0/2044057894 >> 172.21.15.138:0/2044057894 conn(0x559bd5b0ac00 msgr2=0x559bd5c13000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- 172.21.15.138:0/2044057894 shutdown_connections 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- 172.21.15.138:0/2044057894 wait complete. 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f333cb42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b4a8c0 con 0x559bd5b0b000 2024-04-27T05:21:33.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f3339543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f3339543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42496/0 (socket says 172.21.15.138:42496) 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.831+0000 7f3339543640 1 -- 172.21.15.138:0/2803342476 learned_addr learned my addr 172.21.15.138:0/2803342476 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f3339543640 1 -- 172.21.15.138:0/2803342476 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b0f6c0 con 0x559bd5b0b000 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f3339543640 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x559bd69c72c0 tx=0x559bd5c1cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f333653d640 1 -- 172.21.15.138:0/2803342476 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b23e40 con 0x559bd5b0b000 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f333653d640 1 -- 172.21.15.138:0/2803342476 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd69ce000 con 0x559bd5b0b000 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559bd5b0f1e0 con 0x559bd5b0b000 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f333653d640 1 -- 172.21.15.138:0/2803342476 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b234a0 con 0x559bd5b0b000 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.832+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bd5b7cea0 con 0x559bd5b0b000 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333653d640 1 -- 172.21.15.138:0/2803342476 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559bd5b34b00 con 0x559bd5b0b000 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333cb42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333cb42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b2e280 con 0x559bd6902c00 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333653d640 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6903c00 0x559bd5ded080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f333653d640 1 -- 172.21.15.138:0/2803342476 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x559bd5b1da40 con 0x559bd5b0b000 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339543640 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6903c00 0x559bd5ded080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42498/0 (socket says 172.21.15.138:42498) 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339d44640 1 -- 172.21.15.138:0/2769806289 learned_addr learned my addr 172.21.15.138:0/2769806289 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339d44640 1 -- 172.21.15.138:0/2769806289 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b7cd00 con 0x559bd6902c00 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339543640 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6903c00 0x559bd5ded080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x559bd6a12180 tx=0x559bd5c1ccc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3339d44640 1 --2- 172.21.15.138:0/2769806289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x559bd69c7680 tx=0x559bd5c1dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6882fa1f3bdbdd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.833+0000 7f3333d38640 1 -- 172.21.15.138:0/2769806289 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b23e40 con 0x559bd6902c00 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f3333d38640 1 -- 172.21.15.138:0/2769806289 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd69ce600 con 0x559bd6902c00 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f3333d38640 1 -- 172.21.15.138:0/2769806289 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b234a0 con 0x559bd6902c00 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- 172.21.15.138:0/2769806289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 msgr2=0x559bd5decb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 --2- 172.21.15.138:0/2769806289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x559bd69c7680 tx=0x559bd5c1dc20 comp rx=0 tx=0).stop 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- 172.21.15.138:0/2769806289 shutdown_connections 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 --2- 172.21.15.138:0/2769806289 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- 172.21.15.138:0/2769806289 >> 172.21.15.138:0/2769806289 conn(0x559bd5b0b400 msgr2=0x559bd5c15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- 172.21.15.138:0/2769806289 shutdown_connections 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- 172.21.15.138:0/2769806289 wait complete. 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.834+0000 7f333cb42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b2f180 con 0x559bd6902c00 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3339d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3339d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42512/0 (socket says 172.21.15.138:42512) 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 learned_addr learned my addr 172.21.15.138:0/227667153 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b7cb60 con 0x559bd6902c00 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3339d44640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x559bd69c73b0 tx=0x559bd5c1d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3332535640 1 -- 172.21.15.138:0/227667153 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b23e40 con 0x559bd6902c00 2024-04-27T05:21:33.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3332535640 1 -- 172.21.15.138:0/227667153 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd69ce780 con 0x559bd6902c00 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559bd5b7c9c0 con 0x559bd6902c00 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f3332535640 1 -- 172.21.15.138:0/227667153 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5b234a0 con 0x559bd6902c00 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.835+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bd5b7c820 con 0x559bd6902c00 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3332535640 1 -- 172.21.15.138:0/227667153 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559bd5b35600 con 0x559bd6902c00 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3332535640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd5e84000 0x559bd5ded600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3332535640 1 -- 172.21.15.138:0/227667153 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x559bd5b1dc00 con 0x559bd6902c00 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3338d42640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd5e84000 0x559bd5ded600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x559bd6903000 0x559bd5dedb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6903400 con 0x559bd6903000 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3339d44640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x559bd6903000 0x559bd5dedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3338d42640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd5e84000 0x559bd5ded600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x559bd5c17380 tx=0x559bd5e4a120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.836+0000 7f3339d44640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x559bd6903000 0x559bd5dedb80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 realm 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6903800 con 0x559bd6903000 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85000 con 0x559bd6903000 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85400 con 0x559bd6903000 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559bd5e85800 0x559bd5dee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85c00 con 0x559bd5e85800 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.837+0000 7f3339543640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559bd5e85800 0x559bd5dee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f3339543640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559bd5e85800 0x559bd5dee100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f333cb42880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f333cb42880 20 searching for the correct realm 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85000 con 0x559bd5e85800 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f3339543640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559bd5e85000 0x559bd5deec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85c00 con 0x559bd5e85000 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.838+0000 7f3338d42640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559bd5e85000 0x559bd5deec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.839+0000 7f3338d42640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559bd5e85000 0x559bd5deec00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.839+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.839+0000 7f3338d42640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559bd5e85c00 0x559bd5def180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.839+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.839+0000 7f3339d44640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559bd5e85c00 0x559bd5def180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559bd5e85c00 0x559bd5def180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.840+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85000 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.841+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85000 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.842+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.843+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85000 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.844+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.845+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.845+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.845+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.845+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.845+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.845+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85000 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.846+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.846+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85800 2024-04-27T05:21:33.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.846+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.846+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85c00 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.846+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.846+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd6903000 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e84c00 con 0x559bd5e85000 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:33.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85400 con 0x559bd6903000 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6903400 con 0x559bd6903000 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.847+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6903800 con 0x559bd5e85800 2024-04-27T05:21:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 zone default found 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 4 Realm: () 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 zonegroup default 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac000 con 0x559bd5e85c00 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac400 con 0x559bd5e85800 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.848+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac800 con 0x559bd5e85800 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.849+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.849+0000 7f333cb42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.849+0000 7f333cb42880 20 started sync module instance, tier type = 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.849+0000 7f333cb42880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.849+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eacc00 con 0x559bd5e85000 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.850+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.850+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94127796406528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5ead000 con 0x559bd5e85000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.850+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5ead400 con 0x559bd6903000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.851+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94127796406528] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.851+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.851+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94127796407680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5ead800 con 0x559bd6903000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.851+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eadc00 con 0x559bd5e85800 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.852+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94127796407680] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.852+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.852+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94127796408832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac400 con 0x559bd5e85800 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.852+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac800 con 0x559bd5e85000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.853+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.853+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94127796409984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eadc00 con 0x559bd5e85000 2024-04-27T05:21:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.853+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94127796408832] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.853+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac000 con 0x559bd6903000 2024-04-27T05:21:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.854+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.854+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94127796411136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6903800 con 0x559bd6903000 2024-04-27T05:21:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.854+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94127796409984] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.854+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6903400 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.855+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94127796411136] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.855+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.855+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94127796412288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5e85400 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.855+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6a68c00 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.856+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.856+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94127796413440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd6a68800 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.856+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eacc00 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.856+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94127796412288] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94127796413440] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94127796414592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5ead000 con 0x559bd5e85c00 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=0 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=1 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=2 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=3 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=4 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=5 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.857+0000 7f333cb42880 20 add_watcher() i=6 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.858+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94127796414592] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.858+0000 7f333cb42880 20 add_watcher() i=7 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.858+0000 7f333cb42880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.858+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5eac800 con 0x559bd6903000 2024-04-27T05:21:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.858+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.858+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94127796415744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5ead400 con 0x559bd6903000 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94127796415744] v29'62 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f3320511640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f333cb42880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f333cb42880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f333cb42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.859+0000 7f333cb42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b2e780 con 0x559bd5e84c00 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f3338d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f3338d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42526/0 (socket says 172.21.15.138:42526) 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f3338d42640 1 -- 172.21.15.138:0/1969922014 learned_addr learned my addr 172.21.15.138:0/1969922014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f3338d42640 1 -- 172.21.15.138:0/1969922014 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b7c680 con 0x559bd5e84c00 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f3338d42640 1 --2- 172.21.15.138:0/1969922014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x559bd5e66ae0 tx=0x559bd5e4a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494e1d322986583e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f331fd10640 1 -- 172.21.15.138:0/1969922014 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5eae580 con 0x559bd5e84c00 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f331fd10640 1 -- 172.21.15.138:0/1969922014 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd69ce000 con 0x559bd5e84c00 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.860+0000 7f331fd10640 1 -- 172.21.15.138:0/1969922014 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5eae6e0 con 0x559bd5e84c00 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 -- 172.21.15.138:0/1969922014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 msgr2=0x559bd5dee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 --2- 172.21.15.138:0/1969922014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x559bd5e66ae0 tx=0x559bd5e4a180 comp rx=0 tx=0).stop 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 -- 172.21.15.138:0/1969922014 shutdown_connections 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 --2- 172.21.15.138:0/1969922014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 -- 172.21.15.138:0/1969922014 >> 172.21.15.138:0/1969922014 conn(0x559bd5ead800 msgr2=0x559bd5c15800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 -- 172.21.15.138:0/1969922014 shutdown_connections 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 -- 172.21.15.138:0/1969922014 wait complete. 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.861+0000 7f333cb42880 1 Processor -- start 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f333cb42880 1 -- start start 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f333cb42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f333cb42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd5b2e500 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f3338d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f3338d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42534/0 (socket says 172.21.15.138:42534) 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f3338d42640 1 -- 172.21.15.138:0/3240733742 learned_addr learned my addr 172.21.15.138:0/3240733742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f3338d42640 1 -- 172.21.15.138:0/3240733742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd5b7c4e0 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.862+0000 7f3338d42640 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x559bd5e67800 tx=0x559bd5e4aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f331e50d640 1 -- 172.21.15.138:0/3240733742 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5eae840 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f331e50d640 1 -- 172.21.15.138:0/3240733742 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559bd69cea80 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559bd5b0f380 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f331e50d640 1 -- 172.21.15.138:0/3240733742 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559bd5eae580 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bd5b0eea0 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f331e50d640 1 -- 172.21.15.138:0/3240733742 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x559bd5b35b80 con 0x559bd5e84c00 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f331e50d640 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6a68c00 0x559bd5def700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.863+0000 7f331e50d640 1 -- 172.21.15.138:0/3240733742 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x559bd5b1ddc0 con 0x559bd5e84c00 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f3339d44640 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6a68c00 0x559bd5def700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 5 note: GC not initialized 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f331bd08640 20 reqs_thread_entry: start 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f3339d44640 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6a68c00 0x559bd5def700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559bd6a721b0 tx=0x559bd5e4a8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f3319d04640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f33194e2640 20 reqs_thread_entry: start 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f82000 con 0x559bd6903000 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.864+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f82400 con 0x559bd5e85000 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.865+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.865+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f82c00 con 0x559bd6903000 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v29'2 uv2 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f333cb42880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f333cb42880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f333cb42880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94127796418048 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f83000 con 0x559bd6903000 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94127796411136 notify 124554051585 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x559bd5e49d40 con 0x559bd6903000 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3333d38640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051585 cookie 94127796411136 notifier 4325 bl.length()=533 2024-04-27T05:21:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3333d38640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3333d38640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3333d38640 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f96000 con 0x559bd6903000 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(65 notify.4 [notify cookie 94127796418048 out=8b] v0'0 uv33 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.866+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94127796418048 notify 124554051585 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559bd5f0ba00 con 0x559bd6903000 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:33.895 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 1, 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:33.896 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:33.897 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:33.899 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:33.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.867+0000 7f3320511640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.867+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94127796415744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f83400 con 0x559bd6903000 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.868+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94127796415744] v29'63 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.868+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94127796406528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f83800 con 0x559bd5e85000 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.869+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94127796406528] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.869+0000 7f333cb42880 20 remove_watcher() i=0 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.869+0000 7f333cb42880 2 removed watcher, disabling cache 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.869+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94127796407680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f83c00 con 0x559bd6903000 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.870+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94127796407680] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.870+0000 7f333cb42880 20 remove_watcher() i=1 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.870+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94127796408832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f96000 con 0x559bd5e85800 2024-04-27T05:21:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.871+0000 7f3339543640 1 -- 172.21.15.138:0/227667153 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94127796408832] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2fc0 con 0x559bd5e85800 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.871+0000 7f333cb42880 20 remove_watcher() i=2 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.871+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94127796409984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f96800 con 0x559bd5e85000 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.872+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94127796409984] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea3200 con 0x559bd5e85000 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.872+0000 7f333cb42880 20 remove_watcher() i=3 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.872+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94127796411136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f96c00 con 0x559bd6903000 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.873+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94127796411136] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd6903000 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.873+0000 7f333cb42880 20 remove_watcher() i=4 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.873+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94127796412288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f97000 con 0x559bd5e85c00 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.874+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94127796412288] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.874+0000 7f333cb42880 20 remove_watcher() i=5 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.874+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94127796413440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f97400 con 0x559bd5e85c00 2024-04-27T05:21:33.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.875+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94127796413440] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.875+0000 7f333cb42880 20 remove_watcher() i=6 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.875+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94127796414592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559bd5f97800 con 0x559bd5e85c00 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f3339d44640 1 -- 172.21.15.138:0/227667153 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94127796414592] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559bd5ea2900 con 0x559bd5e85c00 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 20 remove_watcher() i=7 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559bd5e85c00 msgr2=0x559bd5def180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559bd5e85c00 0x559bd5def180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x559bd6903000 msgr2=0x559bd5dedb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x559bd6903000 0x559bd5dedb80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559bd5e85000 msgr2=0x559bd5deec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559bd5e85000 0x559bd5deec00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559bd5e85800 msgr2=0x559bd5dee100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559bd5e85800 0x559bd5dee100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd5e84000 msgr2=0x559bd5ded600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd5e84000 0x559bd5ded600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x559bd5c17380 tx=0x559bd5e4a120 comp rx=0 tx=0).stop 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 msgr2=0x559bd5decb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 --2- 172.21.15.138:0/227667153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd6902c00 0x559bd5decb00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x559bd69c73b0 tx=0x559bd5c1d980 comp rx=0 tx=0).stop 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 reap_dead start 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f3338d42640 1 -- 172.21.15.138:0/227667153 reap_dead start 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 shutdown_connections 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 >> 172.21.15.138:0/227667153 conn(0x559bd5b0b400 msgr2=0x559bd5c15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 shutdown_connections 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.876+0000 7f333cb42880 1 -- 172.21.15.138:0/227667153 wait complete. 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6a68c00 msgr2=0x559bd5def700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6a68c00 0x559bd5def700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x559bd6a721b0 tx=0x559bd5e4a8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 msgr2=0x559bd5dee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x559bd5e67800 tx=0x559bd5e4aae0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 shutdown_connections 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6a68c00 0x559bd5def700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/3240733742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5e84c00 0x559bd5dee680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 >> 172.21.15.138:0/3240733742 conn(0x559bd5ead800 msgr2=0x559bd5c15800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 shutdown_connections 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/3240733742 wait complete. 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6903c00 msgr2=0x559bd5ded080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6903c00 0x559bd5ded080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x559bd6a12180 tx=0x559bd5c1ccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 msgr2=0x559bd5dec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x559bd69c72c0 tx=0x559bd5c1cc60 comp rx=0 tx=0).stop 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 shutdown_connections 2024-04-27T05:21:33.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559bd6903c00 0x559bd5ded080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 --2- 172.21.15.138:0/2803342476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559bd5b0b000 0x559bd5dec580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:33.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 >> 172.21.15.138:0/2803342476 conn(0x559bd5b0ac00 msgr2=0x559bd5c13000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:33.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 shutdown_connections 2024-04-27T05:21:33.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.877+0000 7f333cb42880 1 -- 172.21.15.138:0/2803342476 wait complete. 2024-04-27T05:21:33.910 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:33.910 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:33.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:33.911 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b14b42880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5615934b9400 0x56159379a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b14b42880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561593790600 con 0x5615934b9000 2024-04-27T05:21:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b14b42880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5615934f8280 con 0x5615934b9400 2024-04-27T05:21:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b10d42640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5615934b9400 0x56159379a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b10d42640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5615934b9400 0x56159379a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42546/0 (socket says 172.21.15.138:42546) 2024-04-27T05:21:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b11543640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5615934b9000 0x5615935c0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60750/0 (socket says 172.21.15.138:60750) 2024-04-27T05:21:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b10d42640 1 -- 172.21.15.138:0/803223213 learned_addr learned my addr 172.21.15.138:0/803223213 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b10d42640 1 -- 172.21.15.138:0/803223213 >> v1:172.21.15.114:6789/0 conn(0x5615934b9000 legacy=0x5615935c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.986+0000 7f2b10d42640 1 -- 172.21.15.138:0/803223213 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615934bd380 con 0x5615934b9400 2024-04-27T05:21:34.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.988+0000 7f2b10d42640 1 --2- 172.21.15.138:0/803223213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9400 0x56159379a580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5615935c4c00 tx=0x5615935cade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98daed68393b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.988+0000 7f2b10541640 1 -- 172.21.15.138:0/803223213 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d1ce0 con 0x5615934b9400 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.988+0000 7f2b10541640 1 -- 172.21.15.138:0/803223213 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561593791b00 con 0x5615934b9400 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.988+0000 7f2b10541640 1 -- 172.21.15.138:0/803223213 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d1e40 con 0x5615934b9400 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 -- 172.21.15.138:0/803223213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9400 msgr2=0x56159379a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 --2- 172.21.15.138:0/803223213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9400 0x56159379a580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5615935c4c00 tx=0x5615935cade0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 -- 172.21.15.138:0/803223213 shutdown_connections 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 --2- 172.21.15.138:0/803223213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9400 0x56159379a580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 -- 172.21.15.138:0/803223213 >> 172.21.15.138:0/803223213 conn(0x5615934b8c00 msgr2=0x5615935c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 -- 172.21.15.138:0/803223213 shutdown_connections 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 -- 172.21.15.138:0/803223213 wait complete. 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.989+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b14b42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b14b42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934f8780 con 0x5615934b9000 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b11543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b11543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42562/0 (socket says 172.21.15.138:42562) 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b11543640 1 -- 172.21.15.138:0/830171731 learned_addr learned my addr 172.21.15.138:0/830171731 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b11543640 1 -- 172.21.15.138:0/830171731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615934bcea0 con 0x5615934b9000 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b11543640 1 --2- 172.21.15.138:0/830171731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x561594375290 tx=0x5615935cacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7441dc914ee9f60e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b0fd40640 1 -- 172.21.15.138:0/830171731 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d14a0 con 0x5615934b9000 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b0fd40640 1 -- 172.21.15.138:0/830171731 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561593791e00 con 0x5615934b9000 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.990+0000 7f2b0fd40640 1 -- 172.21.15.138:0/830171731 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d1ce0 con 0x5615934b9000 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- 172.21.15.138:0/830171731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 msgr2=0x56159379a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 --2- 172.21.15.138:0/830171731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x561594375290 tx=0x5615935cacc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- 172.21.15.138:0/830171731 shutdown_connections 2024-04-27T05:21:34.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 --2- 172.21.15.138:0/830171731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- 172.21.15.138:0/830171731 >> 172.21.15.138:0/830171731 conn(0x5615934b8c00 msgr2=0x5615935c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- 172.21.15.138:0/830171731 shutdown_connections 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- 172.21.15.138:0/830171731 wait complete. 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b14b42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934f88c0 con 0x5615934b9000 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b11543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b11543640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42578/0 (socket says 172.21.15.138:42578) 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.991+0000 7f2b11543640 1 -- 172.21.15.138:0/3224311078 learned_addr learned my addr 172.21.15.138:0/3224311078 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b11543640 1 -- 172.21.15.138:0/3224311078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615934bd6c0 con 0x5615934b9000 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b11543640 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x561594374b40 tx=0x5615935cac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b0e53d640 1 -- 172.21.15.138:0/3224311078 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d1e40 con 0x5615934b9000 2024-04-27T05:21:34.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b0e53d640 1 -- 172.21.15.138:0/3224311078 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56159438a000 con 0x5615934b9000 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5615934bd1e0 con 0x5615934b9000 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b0e53d640 1 -- 172.21.15.138:0/3224311078 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d14a0 con 0x5615934b9000 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159352aea0 con 0x5615934b9000 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.992+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b14b42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b14b42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934f8c80 con 0x5615942ab800 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b0e53d640 1 -- 172.21.15.138:0/3224311078 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5615934e3600 con 0x5615934b9000 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b0e53d640 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561594416000 0x56159379b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42582/0 (socket says 172.21.15.138:42582) 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b0e53d640 1 -- 172.21.15.138:0/3224311078 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5615934cba40 con 0x5615934b9000 2024-04-27T05:21:34.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11d44640 1 -- 172.21.15.138:0/1552788859 learned_addr learned my addr 172.21.15.138:0/1552788859 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11543640 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561594416000 0x56159379b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11d44640 1 -- 172.21.15.138:0/1552788859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159352ad00 con 0x5615942ab800 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11d44640 1 --2- 172.21.15.138:0/1552788859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x561594375e90 tx=0x5615935caf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab36de0891a0e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b11543640 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561594416000 0x56159379b080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5615943c6450 tx=0x5615935cb4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b0bd38640 1 -- 172.21.15.138:0/1552788859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d1e40 con 0x5615942ab800 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b0bd38640 1 -- 172.21.15.138:0/1552788859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56159438a300 con 0x5615942ab800 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.993+0000 7f2b0bd38640 1 -- 172.21.15.138:0/1552788859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d14a0 con 0x5615942ab800 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- 172.21.15.138:0/1552788859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 msgr2=0x56159379ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 --2- 172.21.15.138:0/1552788859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x561594375e90 tx=0x5615935caf00 comp rx=0 tx=0).stop 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- 172.21.15.138:0/1552788859 shutdown_connections 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 --2- 172.21.15.138:0/1552788859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- 172.21.15.138:0/1552788859 >> 172.21.15.138:0/1552788859 conn(0x5615934b9400 msgr2=0x5615935c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- 172.21.15.138:0/1552788859 shutdown_connections 2024-04-27T05:21:34.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- 172.21.15.138:0/1552788859 wait complete. 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b14b42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934dc280 con 0x5615942ab800 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b11d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b11d44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42598/0 (socket says 172.21.15.138:42598) 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.994+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 learned_addr learned my addr 172.21.15.138:0/3423824187 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159352ab60 con 0x5615942ab800 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5615943759e0 tx=0x5615935cbaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b0a535640 1 -- 172.21.15.138:0/3423824187 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d1e40 con 0x5615942ab800 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b0a535640 1 -- 172.21.15.138:0/3423824187 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56159438a480 con 0x5615942ab800 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56159352a9c0 con 0x5615942ab800 2024-04-27T05:21:34.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b0a535640 1 -- 172.21.15.138:0/3423824187 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615934d14a0 con 0x5615942ab800 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.995+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159352a820 con 0x5615942ab800 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b0a535640 1 -- 172.21.15.138:0/3423824187 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5615934e38c0 con 0x5615942ab800 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b0a535640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56159383c000 0x56159379b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b0a535640 1 -- 172.21.15.138:0/3423824187 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5615934cbc00 con 0x5615942ab800 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56159383c000 0x56159379b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5615942abc00 0x56159379bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383c400 con 0x5615942abc00 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5615942abc00 0x56159379bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56159383c000 0x56159379b600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5615935c4c00 tx=0x561593804120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.996+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5615942abc00 0x56159379bb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 realm 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383c800 con 0x5615942abc00 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383cc00 con 0x5615942abc00 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383d000 con 0x5615942abc00 2024-04-27T05:21:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56159383d400 0x56159379c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383c400 con 0x56159383d400 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.997+0000 7f2b11543640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56159383d400 0x56159379c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b11543640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56159383d400 0x56159379c100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b14b42880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b14b42880 20 searching for the correct realm 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383c800 con 0x56159383d400 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b11543640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56159383c800 0x56159379cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383c400 con 0x56159383c800 2024-04-27T05:21:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.998+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56159383c800 0x56159379cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.999+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56159383c800 0x56159379cc00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.999+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.999+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56159383c400 0x56159379d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.999+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:33.999+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56159383c400 0x56159379d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56159383c400 0x56159379d180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.000+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c800 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.001+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c800 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.002+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.003+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.003+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.003+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.003+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.003+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.003+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.004+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c800 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.005+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c800 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383d400 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c400 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x5615942abc00 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.006+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5615942aa800 con 0x56159383c800 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383cc00 con 0x5615942abc00 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383d000 con 0x5615942abc00 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383d800 con 0x56159383d400 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 zone default found 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 4 Realm: () 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 zonegroup default 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.007+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383dc00 con 0x56159383c400 2024-04-27T05:21:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866000 con 0x56159383d400 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866400 con 0x56159383d400 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 started sync module instance, tier type = 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.008+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866800 con 0x56159383c800 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.009+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.009+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94650668493056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866c00 con 0x56159383c800 2024-04-27T05:21:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.009+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593867000 con 0x5615942abc00 2024-04-27T05:21:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.010+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94650668493056] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.011+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.011+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94650668494208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593867400 con 0x5615942abc00 2024-04-27T05:21:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.011+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593867800 con 0x56159383d400 2024-04-27T05:21:34.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.012+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94650668494208] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.012+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.012+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94650668495360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866000 con 0x56159383d400 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.012+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866400 con 0x56159383c800 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.013+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94650668495360] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.013+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.013+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94650668496512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593867800 con 0x56159383c800 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.013+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383d800 con 0x5615942abc00 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.014+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94650668496512] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.014+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.014+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94650668497664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383cc00 con 0x5615942abc00 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.014+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383d000 con 0x56159383c400 2024-04-27T05:21:34.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.016+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.016+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94650668498816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56159383dc00 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.016+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561594417800 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.016+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94650668497664] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.017+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.017+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94650668499968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866800 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.017+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866c00 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.017+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94650668498816] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.018+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94650668499968] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94650668501120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593866400 con 0x56159383c400 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=0 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=1 2024-04-27T05:21:34.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=2 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=3 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=4 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=5 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=6 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94650668501120] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 20 add_watcher() i=7 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.021+0000 7f2b14b42880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.022+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593867800 con 0x5615942abc00 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.022+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.022+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94650668502272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593867000 con 0x5615942abc00 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94650668502272] v29'64 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2af8511640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b14b42880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b14b42880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b14b42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.023+0000 7f2b14b42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934dc8c0 con 0x561593867c00 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2b10d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2b10d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42602/0 (socket says 172.21.15.138:42602) 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2b10d42640 1 -- 172.21.15.138:0/3582100309 learned_addr learned my addr 172.21.15.138:0/3582100309 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2b10d42640 1 -- 172.21.15.138:0/3582100309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159352a680 con 0x561593867c00 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3582100309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5615943c4930 tx=0x5615938040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115bfd94975a9de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2af7d10640 1 -- 172.21.15.138:0/3582100309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561593868580 con 0x561593867c00 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2af7d10640 1 -- 172.21.15.138:0/3582100309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56159438a000 con 0x561593867c00 2024-04-27T05:21:34.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.024+0000 7f2af7d10640 1 -- 172.21.15.138:0/3582100309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5615938686e0 con 0x561593867c00 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 -- 172.21.15.138:0/3582100309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 msgr2=0x56159379c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3582100309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5615943c4930 tx=0x5615938040c0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 -- 172.21.15.138:0/3582100309 shutdown_connections 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3582100309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 -- 172.21.15.138:0/3582100309 >> 172.21.15.138:0/3582100309 conn(0x561593867400 msgr2=0x5615935c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 -- 172.21.15.138:0/3582100309 shutdown_connections 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 -- 172.21.15.138:0/3582100309 wait complete. 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 Processor -- start 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.025+0000 7f2b14b42880 1 -- start start 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b14b42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b14b42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615934dc780 con 0x561593867c00 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b10d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b10d42640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42610/0 (socket says 172.21.15.138:42610) 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b10d42640 1 -- 172.21.15.138:0/3399984491 learned_addr learned my addr 172.21.15.138:0/3399984491 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b10d42640 1 -- 172.21.15.138:0/3399984491 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159352a4e0 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b10d42640 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x56159381d5c0 tx=0x561593804000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2af650d640 1 -- 172.21.15.138:0/3399984491 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561593868840 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2af650d640 1 -- 172.21.15.138:0/3399984491 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56159438a780 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5615934bcea0 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2af650d640 1 -- 172.21.15.138:0/3399984491 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561593868580 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.026+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615934bd6c0 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.027+0000 7f2af650d640 1 -- 172.21.15.138:0/3399984491 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5615934e2b00 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.027+0000 7f2af650d640 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561593866800 0x56159379d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.027+0000 7f2af650d640 1 -- 172.21.15.138:0/3399984491 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5615934cbdc0 con 0x561593867c00 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.027+0000 7f2b14b42880 5 note: GC not initialized 2024-04-27T05:21:34.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.027+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561593866800 0x56159379d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.027+0000 7f2af3d08640 20 reqs_thread_entry: start 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b14b42880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b11d44640 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561593866800 0x56159379d700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5615943c48d0 tx=0x561593804960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b14b42880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2af1d04640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2af14e2640 20 reqs_thread_entry: start 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b14b42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593928000 con 0x5615942abc00 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.028+0000 7f2b14b42880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 1, 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.050 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.051 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.051 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.051 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.029+0000 7f2af8511640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.029+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94650668502272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593928400 con 0x5615942abc00 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.030+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94650668502272] v29'65 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.030+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94650668493056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593928800 con 0x56159383c800 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.031+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94650668493056] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.031+0000 7f2b14b42880 20 remove_watcher() i=0 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.031+0000 7f2b14b42880 2 removed watcher, disabling cache 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.031+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94650668494208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593928c00 con 0x5615942abc00 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.032+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94650668494208] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.032+0000 7f2b14b42880 20 remove_watcher() i=1 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.032+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94650668495360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593929000 con 0x56159383d400 2024-04-27T05:21:34.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.033+0000 7f2b11543640 1 -- 172.21.15.138:0/3423824187 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94650668495360] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850fc0 con 0x56159383d400 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.033+0000 7f2b14b42880 20 remove_watcher() i=2 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.033+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94650668496512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593929400 con 0x56159383c800 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.034+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94650668496512] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593851200 con 0x56159383c800 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.034+0000 7f2b14b42880 20 remove_watcher() i=3 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.034+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94650668497664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593929800 con 0x5615942abc00 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.035+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94650668497664] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x5615942abc00 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.035+0000 7f2b14b42880 20 remove_watcher() i=4 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.035+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94650668498816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593929c00 con 0x56159383c400 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.036+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94650668498816] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.036+0000 7f2b14b42880 20 remove_watcher() i=5 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.036+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94650668499968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593928000 con 0x56159383c400 2024-04-27T05:21:34.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.037+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94650668499968] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.037+0000 7f2b14b42880 20 remove_watcher() i=6 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.037+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94650668501120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561593928400 con 0x56159383c400 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b11d44640 1 -- 172.21.15.138:0/3423824187 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94650668501120] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561593850900 con 0x56159383c400 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 20 remove_watcher() i=7 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56159383c400 msgr2=0x56159379d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56159383c400 0x56159379d180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5615942abc00 msgr2=0x56159379bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5615942abc00 0x56159379bb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56159383c800 msgr2=0x56159379cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56159383c800 0x56159379cc00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56159383d400 msgr2=0x56159379c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56159383d400 0x56159379c100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56159383c000 msgr2=0x56159379b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56159383c000 0x56159379b600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5615935c4c00 tx=0x561593804120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 msgr2=0x56159379ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3423824187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615942ab800 0x56159379ab00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5615943759e0 tx=0x5615935cbaa0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 reap_dead start 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b10d42640 1 -- 172.21.15.138:0/3423824187 reap_dead start 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 shutdown_connections 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 >> 172.21.15.138:0/3423824187 conn(0x5615934b9400 msgr2=0x5615935c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 shutdown_connections 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3423824187 wait complete. 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561593866800 msgr2=0x56159379d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561593866800 0x56159379d700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5615943c48d0 tx=0x561593804960 comp rx=0 tx=0).stop 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 msgr2=0x56159379c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.038+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x56159381d5c0 tx=0x561593804000 comp rx=0 tx=0).stop 2024-04-27T05:21:34.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 shutdown_connections 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561593866800 0x56159379d700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3399984491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561593867c00 0x56159379c680 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 >> 172.21.15.138:0/3399984491 conn(0x561593867400 msgr2=0x5615935c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 shutdown_connections 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3399984491 wait complete. 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561594416000 msgr2=0x56159379b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561594416000 0x56159379b080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x5615943c6450 tx=0x5615935cb4a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 msgr2=0x56159379a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x561594374b40 tx=0x5615935cac60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 shutdown_connections 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561594416000 0x56159379b080 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 --2- 172.21.15.138:0/3224311078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615934b9000 0x56159379a580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 >> 172.21.15.138:0/3224311078 conn(0x5615934b8c00 msgr2=0x5615935c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 shutdown_connections 2024-04-27T05:21:34.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.039+0000 7f2b14b42880 1 -- 172.21.15.138:0/3224311078 wait complete. 2024-04-27T05:21:34.062 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.062 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-27T05:21:34.062 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-27T05:21:34.062 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-27T05:21:34.139 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.139 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dd3426880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55f2500a5400 0x55f250386580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dd3426880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f25037c600 con 0x55f2500a5000 2024-04-27T05:21:34.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dd3426880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f2500e4280 con 0x55f2500a5400 2024-04-27T05:21:34.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dcf19a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55f2500a5400 0x55f250386580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dcf19a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55f2500a5400 0x55f250386580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42614/0 (socket says 172.21.15.138:42614) 2024-04-27T05:21:34.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dcf19a640 1 -- 172.21.15.138:0/654463625 learned_addr learned my addr 172.21.15.138:0/654463625 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dcf99b640 1 --1- 172.21.15.138:0/654463625 >> v1:172.21.15.114:6789/0 conn(0x55f2500a5000 0x55f2501ac800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60754/0 (socket says 172.21.15.138:60754) 2024-04-27T05:21:34.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dcf19a640 1 -- 172.21.15.138:0/654463625 >> v1:172.21.15.114:6789/0 conn(0x55f2500a5000 legacy=0x55f2501ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.138+0000 7f2dcf19a640 1 -- 172.21.15.138:0/654463625 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2500a9380 con 0x55f2500a5400 2024-04-27T05:21:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.140+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/654463625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5400 0x55f250386580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55f2501b0c00 tx=0x55f2501b6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154b0f639092b4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.140+0000 7f2dce999640 1 -- 172.21.15.138:0/654463625 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bdce0 con 0x55f2500a5400 2024-04-27T05:21:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.140+0000 7f2dce999640 1 -- 172.21.15.138:0/654463625 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f25037db00 con 0x55f2500a5400 2024-04-27T05:21:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.140+0000 7f2dce999640 1 -- 172.21.15.138:0/654463625 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bde40 con 0x55f2500a5400 2024-04-27T05:21:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- 172.21.15.138:0/654463625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5400 msgr2=0x55f250386580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 --2- 172.21.15.138:0/654463625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5400 0x55f250386580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55f2501b0c00 tx=0x55f2501b6de0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- 172.21.15.138:0/654463625 shutdown_connections 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 --2- 172.21.15.138:0/654463625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5400 0x55f250386580 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- 172.21.15.138:0/654463625 >> 172.21.15.138:0/654463625 conn(0x55f2500a4c00 msgr2=0x55f2501ac000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- 172.21.15.138:0/654463625 shutdown_connections 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- 172.21.15.138:0/654463625 wait complete. 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.141+0000 7f2dd3426880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2500e4780 con 0x55f2500a5000 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dcf99b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dcf99b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42626/0 (socket says 172.21.15.138:42626) 2024-04-27T05:21:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dcf99b640 1 -- 172.21.15.138:0/3423132328 learned_addr learned my addr 172.21.15.138:0/3423132328 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dcf99b640 1 -- 172.21.15.138:0/3423132328 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2500a8ea0 con 0x55f2500a5000 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/3423132328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55f250f61290 tx=0x55f2501b6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bbe4d60c6d7922 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dce198640 1 -- 172.21.15.138:0/3423132328 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bd4a0 con 0x55f2500a5000 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dce198640 1 -- 172.21.15.138:0/3423132328 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f25037de00 con 0x55f2500a5000 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.142+0000 7f2dce198640 1 -- 172.21.15.138:0/3423132328 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bdce0 con 0x55f2500a5000 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- 172.21.15.138:0/3423132328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 msgr2=0x55f250386580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3423132328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55f250f61290 tx=0x55f2501b6cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- 172.21.15.138:0/3423132328 shutdown_connections 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3423132328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- 172.21.15.138:0/3423132328 >> 172.21.15.138:0/3423132328 conn(0x55f2500a4c00 msgr2=0x55f2501ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- 172.21.15.138:0/3423132328 shutdown_connections 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- 172.21.15.138:0/3423132328 wait complete. 2024-04-27T05:21:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dd3426880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2500e48c0 con 0x55f2500a5000 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dcf99b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dcf99b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42628/0 (socket says 172.21.15.138:42628) 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.143+0000 7f2dcf99b640 1 -- 172.21.15.138:0/3419337016 learned_addr learned my addr 172.21.15.138:0/3419337016 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dcf99b640 1 -- 172.21.15.138:0/3419337016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2500a96c0 con 0x55f2500a5000 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55f250f60b40 tx=0x55f2501b6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dcc995640 1 -- 172.21.15.138:0/3419337016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bde40 con 0x55f2500a5000 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dcc995640 1 -- 172.21.15.138:0/3419337016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f250f76000 con 0x55f2500a5000 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dcc995640 1 -- 172.21.15.138:0/3419337016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bd4a0 con 0x55f2500a5000 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f2500a91e0 con 0x55f2500a5000 2024-04-27T05:21:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f250116ea0 con 0x55f2500a5000 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.144+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dcc995640 1 -- 172.21.15.138:0/3419337016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f2500cf600 con 0x55f2500a5000 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd3426880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd3426880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2500c8280 con 0x55f250e9d800 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dcc995640 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250ffa000 0x55f250387080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dcc995640 1 -- 172.21.15.138:0/3419337016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55f2500b7a40 con 0x55f2500a5000 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd019c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250ffa000 0x55f250387080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd019c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42630/0 (socket says 172.21.15.138:42630) 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd019c640 1 -- 172.21.15.138:0/765416742 learned_addr learned my addr 172.21.15.138:0/765416742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd019c640 1 -- 172.21.15.138:0/765416742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f250116d00 con 0x55f250e9d800 2024-04-27T05:21:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250ffa000 0x55f250387080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f250fa8a20 tx=0x55f2501b74a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dd019c640 1 --2- 172.21.15.138:0/765416742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f250f612f0 tx=0x55f2501b7c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a63db221a021753 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dca190640 1 -- 172.21.15.138:0/765416742 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bde40 con 0x55f250e9d800 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dca190640 1 -- 172.21.15.138:0/765416742 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f250f76300 con 0x55f250e9d800 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.145+0000 7f2dca190640 1 -- 172.21.15.138:0/765416742 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bd4a0 con 0x55f250e9d800 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- 172.21.15.138:0/765416742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 msgr2=0x55f250386b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 --2- 172.21.15.138:0/765416742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f250f612f0 tx=0x55f2501b7c20 comp rx=0 tx=0).stop 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- 172.21.15.138:0/765416742 shutdown_connections 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 --2- 172.21.15.138:0/765416742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- 172.21.15.138:0/765416742 >> 172.21.15.138:0/765416742 conn(0x55f2500a5400 msgr2=0x55f2501af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- 172.21.15.138:0/765416742 shutdown_connections 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- 172.21.15.138:0/765416742 wait complete. 2024-04-27T05:21:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd3426880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2500c9180 con 0x55f250e9d800 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd019c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd019c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42640/0 (socket says 172.21.15.138:42640) 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.146+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 learned_addr learned my addr 172.21.15.138:0/2998594553 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f250116b60 con 0x55f250e9d800 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dd019c640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55f250f619e0 tx=0x55f2501b7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dc898d640 1 -- 172.21.15.138:0/2998594553 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bde40 con 0x55f250e9d800 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dc898d640 1 -- 172.21.15.138:0/2998594553 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f250f76480 con 0x55f250e9d800 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f2501169c0 con 0x55f250e9d800 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dc898d640 1 -- 172.21.15.138:0/2998594553 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f2500bd4a0 con 0x55f250e9d800 2024-04-27T05:21:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.147+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f250116820 con 0x55f250e9d800 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dc898d640 1 -- 172.21.15.138:0/2998594553 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f2500cf8c0 con 0x55f250e9d800 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dc898d640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250434000 0x55f250387600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dc898d640 1 -- 172.21.15.138:0/2998594553 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55f2500b7c00 con 0x55f250e9d800 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250434000 0x55f250387600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f250e9dc00 0x55f250387b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434400 con 0x55f250e9dc00 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.148+0000 7f2dd019c640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f250e9dc00 0x55f250387b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250434000 0x55f250387600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f2501b0c00 tx=0x55f2503fc120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd019c640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f250e9dc00 0x55f250387b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 realm 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434800 con 0x55f250e9dc00 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434c00 con 0x55f250e9dc00 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435000 con 0x55f250e9dc00 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.149+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f250435400 0x55f250388100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434400 con 0x55f250435400 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f250435400 0x55f250388100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f250435400 0x55f250388100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dd3426880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dd3426880 20 searching for the correct realm 2024-04-27T05:21:34.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.150+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434800 con 0x55f250435400 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf99b640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f250434800 0x55f250388c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434400 con 0x55f250434800 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f250434800 0x55f250388c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f250434800 0x55f250388c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f250434400 0x55f250389180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.151+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.152+0000 7f2dd019c640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f250434400 0x55f250389180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.152+0000 7f2dd019c640 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f250434400 0x55f250389180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.152+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.152+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.152+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.152+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434800 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.153+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.154+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434800 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.155+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.156+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434800 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.157+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434800 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250435400 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.158+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434400 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250e9dc00 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e9c800 con 0x55f250434800 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434c00 con 0x55f250e9dc00 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.159+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435000 con 0x55f250e9dc00 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435800 con 0x55f250435400 2024-04-27T05:21:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 zone default found 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 4 Realm: () 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 zonegroup default 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435c00 con 0x55f250434400 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88000 con 0x55f250435400 2024-04-27T05:21:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.160+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88400 con 0x55f250435400 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.161+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.161+0000 7f2dd3426880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.161+0000 7f2dd3426880 20 started sync module instance, tier type = 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.161+0000 7f2dd3426880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.161+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88800 con 0x55f250434800 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.162+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.162+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94499216288000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88c00 con 0x55f250434800 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.162+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e89000 con 0x55f250e9dc00 2024-04-27T05:21:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.163+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94499216288000] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.163+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.163+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94499216289152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e89400 con 0x55f250e9dc00 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.163+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e89800 con 0x55f250435400 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.164+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94499216289152] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.164+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.164+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94499216290304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88000 con 0x55f250435400 2024-04-27T05:21:34.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.164+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88400 con 0x55f250434800 2024-04-27T05:21:34.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.165+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.165+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94499216291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e89800 con 0x55f250434800 2024-04-27T05:21:34.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.165+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435800 con 0x55f250e9dc00 2024-04-27T05:21:34.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.165+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94499216290304] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.166+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.166+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94499216292608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250434c00 con 0x55f250e9dc00 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.166+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435000 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.166+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94499216291456] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.166+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.167+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94499216293760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250435c00 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.167+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250ffac00 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.167+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94499216292608] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.167+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94499216293760] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.168+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.168+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94499216294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88800 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.168+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88c00 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94499216294912] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94499216296064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e88400 con 0x55f250434400 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=0 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=1 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=2 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=3 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=4 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=5 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=6 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94499216296064] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 20 add_watcher() i=7 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.169+0000 7f2dd3426880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.170+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e89800 con 0x55f250e9dc00 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.170+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.170+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94499216297216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250e89000 con 0x55f250e9dc00 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94499216297216] v29'66 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2db6969640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd3426880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd3426880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd3426880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dd3426880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2500c8780 con 0x55f250e89c00 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.171+0000 7f2dcf19a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2dcf19a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42656/0 (socket says 172.21.15.138:42656) 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2dcf19a640 1 -- 172.21.15.138:0/3952714235 learned_addr learned my addr 172.21.15.138:0/3952714235 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2dcf19a640 1 -- 172.21.15.138:0/3952714235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f250116680 con 0x55f250e89c00 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/3952714235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55f251022b10 tx=0x55f2503fc0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec0bd6704a2d5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2db6168640 1 -- 172.21.15.138:0/3952714235 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f250e8a580 con 0x55f250e89c00 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2db6168640 1 -- 172.21.15.138:0/3952714235 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f250f76000 con 0x55f250e89c00 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.172+0000 7f2db6168640 1 -- 172.21.15.138:0/3952714235 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f250e8a6e0 con 0x55f250e89c00 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- 172.21.15.138:0/3952714235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 msgr2=0x55f250388680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3952714235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55f251022b10 tx=0x55f2503fc0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- 172.21.15.138:0/3952714235 shutdown_connections 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3952714235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- 172.21.15.138:0/3952714235 >> 172.21.15.138:0/3952714235 conn(0x55f250e89400 msgr2=0x55f2501af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- 172.21.15.138:0/3952714235 shutdown_connections 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- 172.21.15.138:0/3952714235 wait complete. 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 Processor -- start 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- start start 2024-04-27T05:21:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dd3426880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2500c8500 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dcf19a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dcf19a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42670/0 (socket says 172.21.15.138:42670) 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.173+0000 7f2dcf19a640 1 -- 172.21.15.138:0/4010423557 learned_addr learned my addr 172.21.15.138:0/4010423557 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2dcf19a640 1 -- 172.21.15.138:0/4010423557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2501164e0 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2dcf19a640 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f250417620 tx=0x55f2503fc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2db4965640 1 -- 172.21.15.138:0/4010423557 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f250e8a840 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2db4965640 1 -- 172.21.15.138:0/4010423557 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f250f76780 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2db4965640 1 -- 172.21.15.138:0/4010423557 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f250e8a580 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f2500a8ea0 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.174+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2500a96c0 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2db4965640 1 -- 172.21.15.138:0/4010423557 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f2500ceb00 con 0x55f250e89c00 2024-04-27T05:21:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2db4965640 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250e88800 0x55f250389700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2db4965640 1 -- 172.21.15.138:0/4010423557 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55f2500b7dc0 con 0x55f250e89c00 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd3426880 5 note: GC not initialized 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd019c640 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250e88800 0x55f250389700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2db2160640 20 reqs_thread_entry: start 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd3426880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd019c640 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250e88800 0x55f250389700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55f250416f30 tx=0x55f2503fc960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd3426880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2db015c640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2daf93a640 20 reqs_thread_entry: start 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd3426880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd3426880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.175+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051a000 con 0x55f250e9dc00 2024-04-27T05:21:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd3426880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd3426880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd3426880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd3426880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051a400 con 0x55f250434800 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.176+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051ac00 con 0x55f250e9dc00 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.177+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v29'3 uv3 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.177+0000 7f2dd3426880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.177+0000 7f2dd3426880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.177+0000 7f2dd3426880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.177+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94499216299520 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051b000 con 0x55f250e9dc00 2024-04-27T05:21:34.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94499216292608 notify 124554051586 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55f2503fbd40 con 0x55f250e9dc00 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dca190640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051586 cookie 94499216292608 notifier 4341 bl.length()=533 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(65 notify.4 [notify cookie 94499216299520 out=8b] v0'0 uv39 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dca190640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dca190640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dca190640 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250530000 con 0x55f250e9dc00 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.178+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94499216299520 notify 124554051586 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f2504a1860 con 0x55f250e9dc00 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.203 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:34.204 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.205 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.206 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.179+0000 7f2db6969640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.179+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94499216297216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051b400 con 0x55f250e9dc00 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.179+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94499216297216] v29'67 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.179+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94499216288000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051b800 con 0x55f250434800 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.180+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94499216288000] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.180+0000 7f2dd3426880 20 remove_watcher() i=0 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.180+0000 7f2dd3426880 2 removed watcher, disabling cache 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.180+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94499216289152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051bc00 con 0x55f250e9dc00 2024-04-27T05:21:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.181+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94499216289152] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.181+0000 7f2dd3426880 20 remove_watcher() i=1 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.181+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94499216290304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250530000 con 0x55f250435400 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.182+0000 7f2dcf99b640 1 -- 172.21.15.138:0/2998594553 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94499216290304] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044afc0 con 0x55f250435400 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.182+0000 7f2dd3426880 20 remove_watcher() i=2 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.182+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94499216291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250530800 con 0x55f250434800 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.183+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94499216291456] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044b200 con 0x55f250434800 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.183+0000 7f2dd3426880 20 remove_watcher() i=3 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.183+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94499216292608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250530c00 con 0x55f250e9dc00 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.184+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94499216292608] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250e9dc00 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.184+0000 7f2dd3426880 20 remove_watcher() i=4 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.184+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94499216293760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250531000 con 0x55f250434400 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.185+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94499216293760] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.185+0000 7f2dd3426880 20 remove_watcher() i=5 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.185+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94499216294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f250531400 con 0x55f250434400 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.186+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94499216294912] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.186+0000 7f2dd3426880 20 remove_watcher() i=6 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.186+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94499216296064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f25051a000 con 0x55f250434400 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd019c640 1 -- 172.21.15.138:0/2998594553 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94499216296064] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f25044a900 con 0x55f250434400 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 20 remove_watcher() i=7 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f250434400 msgr2=0x55f250389180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f250434400 0x55f250389180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f250e9dc00 msgr2=0x55f250387b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f250e9dc00 0x55f250387b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f250434800 msgr2=0x55f250388c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f250434800 0x55f250388c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f250435400 msgr2=0x55f250388100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f250435400 0x55f250388100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250434000 msgr2=0x55f250387600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250434000 0x55f250387600 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f2501b0c00 tx=0x55f2503fc120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 msgr2=0x55f250386b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 --2- 172.21.15.138:0/2998594553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e9d800 0x55f250386b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55f250f619e0 tx=0x55f2501b7a40 comp rx=0 tx=0).stop 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 reap_dead start 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dcf19a640 1 -- 172.21.15.138:0/2998594553 reap_dead start 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 shutdown_connections 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 >> 172.21.15.138:0/2998594553 conn(0x55f2500a5400 msgr2=0x55f2501af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 shutdown_connections 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.187+0000 7f2dd3426880 1 -- 172.21.15.138:0/2998594553 wait complete. 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250e88800 msgr2=0x55f250389700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250e88800 0x55f250389700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55f250416f30 tx=0x55f2503fc960 comp rx=0 tx=0).stop 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 msgr2=0x55f250388680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f250417620 tx=0x55f2503fc000 comp rx=0 tx=0).stop 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 shutdown_connections 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250e88800 0x55f250389700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/4010423557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f250e89c00 0x55f250388680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 >> 172.21.15.138:0/4010423557 conn(0x55f250e89400 msgr2=0x55f2501af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 shutdown_connections 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/4010423557 wait complete. 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250ffa000 msgr2=0x55f250387080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250ffa000 0x55f250387080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f250fa8a20 tx=0x55f2501b74a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 msgr2=0x55f250386580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55f250f60b40 tx=0x55f2501b6c60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 shutdown_connections 2024-04-27T05:21:34.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f250ffa000 0x55f250387080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 --2- 172.21.15.138:0/3419337016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2500a5000 0x55f250386580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 >> 172.21.15.138:0/3419337016 conn(0x55f2500a4c00 msgr2=0x55f2501ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 shutdown_connections 2024-04-27T05:21:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.188+0000 7f2dd3426880 1 -- 172.21.15.138:0/3419337016 wait complete. 2024-04-27T05:21:34.217 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.217 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:34.217 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:34.217 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:34.295 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.295 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d8298880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c246847000 0x55c246b28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d8298880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c246b1e600 con 0x55c246847400 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d8298880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c246886280 con 0x55c246847000 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d6a44640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c246847000 0x55c246b28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d6a44640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c246847000 0x55c246b28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42676/0 (socket says 172.21.15.138:42676) 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d6243640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55c246847400 0x55c24694e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60766/0 (socket says 172.21.15.138:60766) 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.293+0000 7f70d6a44640 1 -- 172.21.15.138:0/1216273083 learned_addr learned my addr 172.21.15.138:0/1216273083 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.294+0000 7f70d6a44640 1 -- 172.21.15.138:0/1216273083 >> v1:172.21.15.114:6789/0 conn(0x55c246847400 legacy=0x55c24694e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.294+0000 7f70d6a44640 1 -- 172.21.15.138:0/1216273083 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24684b380 con 0x55c246847000 2024-04-27T05:21:34.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.295+0000 7f70d6a44640 1 --2- 172.21.15.138:0/1216273083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55c246952d50 tx=0x55c246958de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9af83e2fd3d1ade server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.295+0000 7f70d5a42640 1 -- 172.21.15.138:0/1216273083 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685fce0 con 0x55c246847000 2024-04-27T05:21:34.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.295+0000 7f70d5a42640 1 -- 172.21.15.138:0/1216273083 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c246b1fb00 con 0x55c246847000 2024-04-27T05:21:34.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.295+0000 7f70d5a42640 1 -- 172.21.15.138:0/1216273083 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685fe40 con 0x55c246847000 2024-04-27T05:21:34.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 -- 172.21.15.138:0/1216273083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 msgr2=0x55c246b28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 --2- 172.21.15.138:0/1216273083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55c246952d50 tx=0x55c246958de0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 -- 172.21.15.138:0/1216273083 shutdown_connections 2024-04-27T05:21:34.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 --2- 172.21.15.138:0/1216273083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 -- 172.21.15.138:0/1216273083 >> 172.21.15.138:0/1216273083 conn(0x55c246846c00 msgr2=0x55c24694e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 -- 172.21.15.138:0/1216273083 shutdown_connections 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.296+0000 7f70d8298880 1 -- 172.21.15.138:0/1216273083 wait complete. 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d8298880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d8298880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c246886780 con 0x55c246847000 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d6a44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d6a44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42682/0 (socket says 172.21.15.138:42682) 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.297+0000 7f70d6a44640 1 -- 172.21.15.138:0/1954104199 learned_addr learned my addr 172.21.15.138:0/1954104199 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d6a44640 1 -- 172.21.15.138:0/1954104199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24684aea0 con 0x55c246847000 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d6a44640 1 --2- 172.21.15.138:0/1954104199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55c24764b560 tx=0x55c246958cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b43414cebdcfd87 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d5241640 1 -- 172.21.15.138:0/1954104199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685f4a0 con 0x55c246847000 2024-04-27T05:21:34.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d5241640 1 -- 172.21.15.138:0/1954104199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c246b1fe00 con 0x55c246847000 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d5241640 1 -- 172.21.15.138:0/1954104199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685fce0 con 0x55c246847000 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 -- 172.21.15.138:0/1954104199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 msgr2=0x55c246b28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 --2- 172.21.15.138:0/1954104199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55c24764b560 tx=0x55c246958cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 -- 172.21.15.138:0/1954104199 shutdown_connections 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 --2- 172.21.15.138:0/1954104199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 -- 172.21.15.138:0/1954104199 >> 172.21.15.138:0/1954104199 conn(0x55c246846c00 msgr2=0x55c24694f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 -- 172.21.15.138:0/1954104199 shutdown_connections 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.298+0000 7f70d8298880 1 -- 172.21.15.138:0/1954104199 wait complete. 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d8298880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d8298880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2468868c0 con 0x55c246847000 2024-04-27T05:21:34.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d6a44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d6a44640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42684/0 (socket says 172.21.15.138:42684) 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d6a44640 1 -- 172.21.15.138:0/254365276 learned_addr learned my addr 172.21.15.138:0/254365276 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d6a44640 1 -- 172.21.15.138:0/254365276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24684b6c0 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.299+0000 7f70d6a44640 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c2477032c0 tx=0x55c246958c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d3a3e640 1 -- 172.21.15.138:0/254365276 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685fe40 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d3a3e640 1 -- 172.21.15.138:0/254365276 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c247626000 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c24684b1e0 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d3a3e640 1 -- 172.21.15.138:0/254365276 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685f4a0 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2468b8ea0 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d3a3e640 1 -- 172.21.15.138:0/254365276 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c246870b00 con 0x55c246847000 2024-04-27T05:21:34.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d8298880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d8298880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c246886c80 con 0x55c24763ec00 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d3a3e640 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c24763fc00 0x55c246b29080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.300+0000 7f70d3a3e640 1 -- 172.21.15.138:0/254365276 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c246859a40 con 0x55c246847000 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d7245640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d6a44640 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c24763fc00 0x55c246b29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d7245640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42694/0 (socket says 172.21.15.138:42694) 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d7245640 1 -- 172.21.15.138:0/1607894113 learned_addr learned my addr 172.21.15.138:0/1607894113 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d7245640 1 -- 172.21.15.138:0/1607894113 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2468b8d00 con 0x55c24763ec00 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d6a44640 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c24763fc00 0x55c246b29080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55c24773e780 tx=0x55c246958cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d7245640 1 --2- 172.21.15.138:0/1607894113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55c247703650 tx=0x55c246959c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e89741b2a29174 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d1239640 1 -- 172.21.15.138:0/1607894113 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685fe40 con 0x55c24763ec00 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d1239640 1 -- 172.21.15.138:0/1607894113 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c247626600 con 0x55c24763ec00 2024-04-27T05:21:34.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d1239640 1 -- 172.21.15.138:0/1607894113 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685f4a0 con 0x55c24763ec00 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d8298880 1 -- 172.21.15.138:0/1607894113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 msgr2=0x55c246b28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.301+0000 7f70d8298880 1 --2- 172.21.15.138:0/1607894113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55c247703650 tx=0x55c246959c20 comp rx=0 tx=0).stop 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 -- 172.21.15.138:0/1607894113 shutdown_connections 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 --2- 172.21.15.138:0/1607894113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 -- 172.21.15.138:0/1607894113 >> 172.21.15.138:0/1607894113 conn(0x55c246847400 msgr2=0x55c246951000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 -- 172.21.15.138:0/1607894113 shutdown_connections 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 -- 172.21.15.138:0/1607894113 wait complete. 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d8298880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24686a280 con 0x55c24763ec00 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d7245640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d7245640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42702/0 (socket says 172.21.15.138:42702) 2024-04-27T05:21:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 learned_addr learned my addr 172.21.15.138:0/1712943596 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2468b8b60 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.302+0000 7f70d7245640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55c247702b40 tx=0x55c246959980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.303+0000 7f70cfa36640 1 -- 172.21.15.138:0/1712943596 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685fe40 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.303+0000 7f70cfa36640 1 -- 172.21.15.138:0/1712943596 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c247626780 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.303+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2468b89c0 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.303+0000 7f70cfa36640 1 -- 172.21.15.138:0/1712943596 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c24685f4a0 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.303+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2468b8820 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.303+0000 7f70cfa36640 1 -- 172.21.15.138:0/1712943596 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c246871600 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70cfa36640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c246bca000 0x55c246b29600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70cfa36640 1 -- 172.21.15.138:0/1712943596 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c246859c00 con 0x55c24763ec00 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d6243640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c246bca000 0x55c246b29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c24763f000 0x55c246b29b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c24763f400 con 0x55c24763f000 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d7245640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c24763f000 0x55c246b29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d6243640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c246bca000 0x55c246b29600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55c246953380 tx=0x55c246b8c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d7245640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c24763f000 0x55c246b29b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 20 realm 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.304+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c24763f800 con 0x55c24763f000 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcb000 con 0x55c24763f000 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcb400 con 0x55c24763f000 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c246bcb800 0x55c246b2a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcbc00 con 0x55c246bcb800 2024-04-27T05:21:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.305+0000 7f70d6a44640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c246bcb800 0x55c246b2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d6a44640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c246bcb800 0x55c246b2a100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d8298880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d8298880 20 searching for the correct realm 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcb000 con 0x55c246bcb800 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d6a44640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c246bcb000 0x55c246b2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcbc00 con 0x55c246bcb000 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.306+0000 7f70d6243640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c246bcb000 0x55c246b2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.307+0000 7f70d6243640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c246bcb000 0x55c246b2ac00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.307+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.307+0000 7f70d6243640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c246bcbc00 0x55c246b2b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.307+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.307+0000 7f70d7245640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c246bcbc00 0x55c246b2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.307+0000 7f70d7245640 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c246bcbc00 0x55c246b2b180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.308+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb000 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.309+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb000 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.310+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.311+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.312+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.312+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.312+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.312+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.313+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb000 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb800 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcbc00 2024-04-27T05:21:34.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c24763f000 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.314+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcac00 con 0x55c246bcb000 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcb400 con 0x55c24763f000 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c24763f400 con 0x55c24763f000 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c24763f800 con 0x55c246bcb800 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 zone default found 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 4 Realm: () 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 zonegroup default 2024-04-27T05:21:34.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.315+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec000 con 0x55c246bcbc00 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec400 con 0x55c246bcb800 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec800 con 0x55c246bcb800 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 started sync module instance, tier type = 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.316+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246becc00 con 0x55c246bcb000 2024-04-27T05:21:34.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.317+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.317+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94292898089216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bed000 con 0x55c246bcb000 2024-04-27T05:21:34.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.317+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bed400 con 0x55c24763f000 2024-04-27T05:21:34.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.318+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94292898089216] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.319+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.319+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94292898090368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bed800 con 0x55c24763f000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.319+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bedc00 con 0x55c246bcb800 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.320+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.320+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94292898091520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec400 con 0x55c246bcb800 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.320+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec800 con 0x55c246bcb000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.320+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94292898090368] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.321+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.321+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94292898092672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bedc00 con 0x55c246bcb000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.321+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec000 con 0x55c24763f000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.321+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94292898091520] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.322+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.322+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94292898093824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c24763f800 con 0x55c24763f000 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.322+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c24763f400 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.322+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94292898092672] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.323+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.323+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94292898094976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bcb400 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.323+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94292898093824] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.323+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c2477a6c00 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.324+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.324+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94292898094976] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.324+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94292898096128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c2477a6800 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.324+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246becc00 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94292898096128] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94292898097280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bed000 con 0x55c246bcbc00 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=0 2024-04-27T05:21:34.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=1 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=2 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=3 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=4 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=5 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=6 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94292898097280] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 20 add_watcher() i=7 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.325+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bec800 con 0x55c24763f000 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.326+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.326+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94292898098432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246bed400 con 0x55c24763f000 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94292898098432] v29'68 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70bda12640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d8298880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d8298880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d8298880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d8298880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24686a8c0 con 0x55c246bcac00 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d6243640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d6243640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42718/0 (socket says 172.21.15.138:42718) 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.327+0000 7f70d6243640 1 -- 172.21.15.138:0/528899864 learned_addr learned my addr 172.21.15.138:0/528899864 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.328+0000 7f70d6243640 1 -- 172.21.15.138:0/528899864 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2468b8680 con 0x55c246bcac00 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.328+0000 7f70d6243640 1 --2- 172.21.15.138:0/528899864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55c246ba8ae0 tx=0x55c246b8c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9427779705c706e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.328+0000 7f70bd211640 1 -- 172.21.15.138:0/528899864 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c246bee580 con 0x55c246bcac00 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.328+0000 7f70bd211640 1 -- 172.21.15.138:0/528899864 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c247626000 con 0x55c246bcac00 2024-04-27T05:21:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.328+0000 7f70bd211640 1 -- 172.21.15.138:0/528899864 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c246bee6e0 con 0x55c246bcac00 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- 172.21.15.138:0/528899864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 msgr2=0x55c246b2a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 --2- 172.21.15.138:0/528899864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55c246ba8ae0 tx=0x55c246b8c180 comp rx=0 tx=0).stop 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- 172.21.15.138:0/528899864 shutdown_connections 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 --2- 172.21.15.138:0/528899864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- 172.21.15.138:0/528899864 >> 172.21.15.138:0/528899864 conn(0x55c246bed800 msgr2=0x55c246951800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- 172.21.15.138:0/528899864 shutdown_connections 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- 172.21.15.138:0/528899864 wait complete. 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 Processor -- start 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- start start 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d8298880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24686a780 con 0x55c246bcac00 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.329+0000 7f70d6243640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70d6243640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42730/0 (socket says 172.21.15.138:42730) 2024-04-27T05:21:34.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70d6243640 1 -- 172.21.15.138:0/3367586674 learned_addr learned my addr 172.21.15.138:0/3367586674 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70d6243640 1 -- 172.21.15.138:0/3367586674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2468b84e0 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70d6243640 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55c246ba9860 tx=0x55c246b8cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70bba0e640 1 -- 172.21.15.138:0/3367586674 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c246bee840 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70bba0e640 1 -- 172.21.15.138:0/3367586674 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c247626a80 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70bba0e640 1 -- 172.21.15.138:0/3367586674 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c246bee580 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c24684b380 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.330+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c24684aea0 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70bba0e640 1 -- 172.21.15.138:0/3367586674 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c246871b80 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70bba0e640 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2477a6c00 0x55c246b2b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70bba0e640 1 -- 172.21.15.138:0/3367586674 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c246859dc0 con 0x55c246bcac00 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70d7245640 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2477a6c00 0x55c246b2b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70d8298880 5 note: GC not initialized 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70b9209640 20 reqs_thread_entry: start 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70d8298880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70d7245640 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2477a6c00 0x55c246b2b700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55c2477a4c30 tx=0x55c246b8c8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70d8298880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.331+0000 7f70b7205640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.332+0000 7f70b69e3640 20 reqs_thread_entry: start 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.332+0000 7f70d8298880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.332+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cba000 con 0x55c24763f000 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.332+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.332+0000 7f70d8298880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.353 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.354 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.355 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.333+0000 7f70bda12640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.333+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94292898098432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cba400 con 0x55c24763f000 2024-04-27T05:21:34.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.334+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94292898098432] v29'69 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.334+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94292898089216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cba800 con 0x55c246bcb000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.335+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94292898089216] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.335+0000 7f70d8298880 20 remove_watcher() i=0 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.335+0000 7f70d8298880 2 removed watcher, disabling cache 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.335+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94292898090368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cbac00 con 0x55c24763f000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.336+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94292898090368] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.336+0000 7f70d8298880 20 remove_watcher() i=1 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.336+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94292898091520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cbb000 con 0x55c246bcb800 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.337+0000 7f70d6a44640 1 -- 172.21.15.138:0/1712943596 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94292898091520] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdefc0 con 0x55c246bcb800 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.337+0000 7f70d8298880 20 remove_watcher() i=2 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.337+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94292898092672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cbb400 con 0x55c246bcb000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.337+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94292898092672] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bdf200 con 0x55c246bcb000 2024-04-27T05:21:34.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.337+0000 7f70d8298880 20 remove_watcher() i=3 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.337+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94292898093824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cbb800 con 0x55c24763f000 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.338+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94292898093824] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c24763f000 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.338+0000 7f70d8298880 20 remove_watcher() i=4 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.338+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94292898094976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cbbc00 con 0x55c246bcbc00 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.339+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94292898094976] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.339+0000 7f70d8298880 20 remove_watcher() i=5 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.339+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94292898096128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cba000 con 0x55c246bcbc00 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.340+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94292898096128] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.340+0000 7f70d8298880 20 remove_watcher() i=6 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.340+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94292898097280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c246cba400 con 0x55c246bcbc00 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d7245640 1 -- 172.21.15.138:0/1712943596 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94292898097280] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c246bde900 con 0x55c246bcbc00 2024-04-27T05:21:34.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 20 remove_watcher() i=7 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c246bcbc00 msgr2=0x55c246b2b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c246bcbc00 0x55c246b2b180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c24763f000 msgr2=0x55c246b29b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c24763f000 0x55c246b29b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c246bcb000 msgr2=0x55c246b2ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c246bcb000 0x55c246b2ac00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c246bcb800 msgr2=0x55c246b2a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c246bcb800 0x55c246b2a100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c246bca000 msgr2=0x55c246b29600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c246bca000 0x55c246b29600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55c246953380 tx=0x55c246b8c120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 msgr2=0x55c246b28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 --2- 172.21.15.138:0/1712943596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c24763ec00 0x55c246b28b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55c247702b40 tx=0x55c246959980 comp rx=0 tx=0).stop 2024-04-27T05:21:34.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 reap_dead start 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d6243640 1 -- 172.21.15.138:0/1712943596 reap_dead start 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 shutdown_connections 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 >> 172.21.15.138:0/1712943596 conn(0x55c246847400 msgr2=0x55c246951000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 shutdown_connections 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.341+0000 7f70d8298880 1 -- 172.21.15.138:0/1712943596 wait complete. 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2477a6c00 msgr2=0x55c246b2b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2477a6c00 0x55c246b2b700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55c2477a4c30 tx=0x55c246b8c8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 msgr2=0x55c246b2a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55c246ba9860 tx=0x55c246b8cae0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 shutdown_connections 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2477a6c00 0x55c246b2b700 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/3367586674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246bcac00 0x55c246b2a680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 >> 172.21.15.138:0/3367586674 conn(0x55c246bed800 msgr2=0x55c246951800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 shutdown_connections 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/3367586674 wait complete. 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c24763fc00 msgr2=0x55c246b29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c24763fc00 0x55c246b29080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55c24773e780 tx=0x55c246958cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 msgr2=0x55c246b28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c2477032c0 tx=0x55c246958c60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 shutdown_connections 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c24763fc00 0x55c246b29080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 --2- 172.21.15.138:0/254365276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c246847000 0x55c246b28580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 >> 172.21.15.138:0/254365276 conn(0x55c246846c00 msgr2=0x55c24694f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 shutdown_connections 2024-04-27T05:21:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.342+0000 7f70d8298880 1 -- 172.21.15.138:0/254365276 wait complete. 2024-04-27T05:21:34.367 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.368 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-27T05:21:34.368 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-27T05:21:34.368 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-04-27T05:21:34.450 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.450 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.447+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.447+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.447+0000 7fc2d2533880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a244183400 0x55a244464580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.447+0000 7fc2d2533880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a24445a600 con 0x55a244183000 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2d2533880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a2441c2280 con 0x55a244183400 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2ce621640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a244183400 0x55a244464580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2cee22640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a244183000 0x55a24428a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60780/0 (socket says 172.21.15.138:60780) 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2ce621640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a244183400 0x55a244464580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42746/0 (socket says 172.21.15.138:42746) 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2cee22640 1 -- 172.21.15.138:0/1612713828 learned_addr learned my addr 172.21.15.138:0/1612713828 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2ce621640 1 -- 172.21.15.138:0/1612713828 >> v1:172.21.15.114:6789/0 conn(0x55a244183000 legacy=0x55a24428a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.448+0000 7fc2ce621640 1 -- 172.21.15.138:0/1612713828 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a244187380 con 0x55a244183400 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2ce621640 1 --2- 172.21.15.138:0/1612713828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183400 0x55a244464580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55a24428ec00 tx=0x55a244294de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d00bfb0448b5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2cde20640 1 -- 172.21.15.138:0/1612713828 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419bce0 con 0x55a244183400 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2cde20640 1 -- 172.21.15.138:0/1612713828 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a24445bb00 con 0x55a244183400 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2cde20640 1 -- 172.21.15.138:0/1612713828 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419be40 con 0x55a244183400 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 -- 172.21.15.138:0/1612713828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183400 msgr2=0x55a244464580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 --2- 172.21.15.138:0/1612713828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183400 0x55a244464580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55a24428ec00 tx=0x55a244294de0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 -- 172.21.15.138:0/1612713828 shutdown_connections 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 --2- 172.21.15.138:0/1612713828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183400 0x55a244464580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 -- 172.21.15.138:0/1612713828 >> 172.21.15.138:0/1612713828 conn(0x55a244182c00 msgr2=0x55a24428a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 -- 172.21.15.138:0/1612713828 shutdown_connections 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.450+0000 7fc2d2533880 1 -- 172.21.15.138:0/1612713828 wait complete. 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2d2533880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2d2533880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2441c2780 con 0x55a244183000 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2cee22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2cee22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42750/0 (socket says 172.21.15.138:42750) 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2cee22640 1 -- 172.21.15.138:0/2797455770 learned_addr learned my addr 172.21.15.138:0/2797455770 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.451+0000 7fc2cee22640 1 -- 172.21.15.138:0/2797455770 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a244186ea0 con 0x55a244183000 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2cee22640 1 --2- 172.21.15.138:0/2797455770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55a24503ecc0 tx=0x55a244294cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40c8b2256d659e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2cd61f640 1 -- 172.21.15.138:0/2797455770 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419b4a0 con 0x55a244183000 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2cd61f640 1 -- 172.21.15.138:0/2797455770 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a24445be00 con 0x55a244183000 2024-04-27T05:21:34.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2cd61f640 1 -- 172.21.15.138:0/2797455770 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419bce0 con 0x55a244183000 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 -- 172.21.15.138:0/2797455770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 msgr2=0x55a244464580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2797455770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55a24503ecc0 tx=0x55a244294cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 -- 172.21.15.138:0/2797455770 shutdown_connections 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2797455770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 -- 172.21.15.138:0/2797455770 >> 172.21.15.138:0/2797455770 conn(0x55a244182c00 msgr2=0x55a24428b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 -- 172.21.15.138:0/2797455770 shutdown_connections 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 -- 172.21.15.138:0/2797455770 wait complete. 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.452+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2d2533880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2d2533880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2441c28c0 con 0x55a244183000 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cee22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cee22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42762/0 (socket says 172.21.15.138:42762) 2024-04-27T05:21:34.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cee22640 1 -- 172.21.15.138:0/322957503 learned_addr learned my addr 172.21.15.138:0/322957503 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cee22640 1 -- 172.21.15.138:0/322957503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2441876c0 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cee22640 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55a24503eed0 tx=0x55a244294c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cbe1c640 1 -- 172.21.15.138:0/322957503 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419be40 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cbe1c640 1 -- 172.21.15.138:0/322957503 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a245054000 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a2441871e0 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.453+0000 7fc2cbe1c640 1 -- 172.21.15.138:0/322957503 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419b4a0 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2441f4ea0 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cbe1c640 1 -- 172.21.15.138:0/322957503 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a2441ad600 con 0x55a244183000 2024-04-27T05:21:34.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2d2533880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2d2533880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2441a6280 con 0x55a244f7b800 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cbe1c640 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a2450da000 0x55a244465080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cbe1c640 1 -- 172.21.15.138:0/322957503 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a244195a40 con 0x55a244183000 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cf623640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cf623640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42766/0 (socket says 172.21.15.138:42766) 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cee22640 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a2450da000 0x55a244465080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.454+0000 7fc2cf623640 1 -- 172.21.15.138:0/1140622852 learned_addr learned my addr 172.21.15.138:0/1140622852 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2cf623640 1 -- 172.21.15.138:0/1140622852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2441f4d00 con 0x55a244f7b800 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2cee22640 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a2450da000 0x55a244465080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55a245084450 tx=0x55a2442954a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2cf623640 1 --2- 172.21.15.138:0/1140622852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a24503f440 tx=0x55a244294c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed6902df5d10ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2c9617640 1 -- 172.21.15.138:0/1140622852 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419be40 con 0x55a244f7b800 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2c9617640 1 -- 172.21.15.138:0/1140622852 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a245054300 con 0x55a244f7b800 2024-04-27T05:21:34.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2c9617640 1 -- 172.21.15.138:0/1140622852 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419b4a0 con 0x55a244f7b800 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 -- 172.21.15.138:0/1140622852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 msgr2=0x55a244464b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 --2- 172.21.15.138:0/1140622852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a24503f440 tx=0x55a244294c00 comp rx=0 tx=0).stop 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 -- 172.21.15.138:0/1140622852 shutdown_connections 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 --2- 172.21.15.138:0/1140622852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 -- 172.21.15.138:0/1140622852 >> 172.21.15.138:0/1140622852 conn(0x55a244183400 msgr2=0x55a24428d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 -- 172.21.15.138:0/1140622852 shutdown_connections 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.455+0000 7fc2d2533880 1 -- 172.21.15.138:0/1140622852 wait complete. 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2d2533880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2d2533880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2441a7180 con 0x55a244f7b800 2024-04-27T05:21:34.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2cf623640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2cf623640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42768/0 (socket says 172.21.15.138:42768) 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 learned_addr learned my addr 172.21.15.138:0/2168965339 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2441f4b60 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2cf623640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55a24503f950 tx=0x55a244295a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2c7e14640 1 -- 172.21.15.138:0/2168965339 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419be40 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2c7e14640 1 -- 172.21.15.138:0/2168965339 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a245054480 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a2441f49c0 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2c7e14640 1 -- 172.21.15.138:0/2168965339 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a24419b4a0 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.456+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2441f4820 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2c7e14640 1 -- 172.21.15.138:0/2168965339 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a2441ad8c0 con 0x55a244f7b800 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2c7e14640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244512000 0x55a244465600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2c7e14640 1 -- 172.21.15.138:0/2168965339 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a244195c00 con 0x55a244f7b800 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2ce621640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244512000 0x55a244465600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a244f7bc00 0x55a244465b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.457+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512400 con 0x55a244f7bc00 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2cf623640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a244f7bc00 0x55a244465b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2ce621640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244512000 0x55a244465600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a24428ec00 tx=0x55a2444da120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2cf623640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a244f7bc00 0x55a244465b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 realm 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512800 con 0x55a244f7bc00 2024-04-27T05:21:34.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512c00 con 0x55a244f7bc00 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.458+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513000 con 0x55a244f7bc00 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a244513400 0x55a244466100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512400 con 0x55a244513400 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2cee22640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a244513400 0x55a244466100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2cee22640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a244513400 0x55a244466100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 20 searching for the correct realm 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.459+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512800 con 0x55a244513400 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2cee22640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a244512800 0x55a244466c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512400 con 0x55a244512800 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2ce621640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a244512800 0x55a244466c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2ce621640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a244512800 0x55a244466c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2ce621640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a244512400 0x55a244467180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.460+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.461+0000 7fc2cf623640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a244512400 0x55a244467180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.461+0000 7fc2cf623640 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a244512400 0x55a244467180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.461+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.461+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.461+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.461+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512800 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.462+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512800 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.463+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.464+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.464+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.464+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.464+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.464+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.464+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.465+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512800 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.466+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.467+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.467+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512800 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.467+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.467+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244513400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.467+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.467+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512400 2024-04-27T05:21:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244f7bc00 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f7a800 con 0x55a244512800 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512c00 con 0x55a244f7bc00 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513000 con 0x55a244f7bc00 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.468+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513800 con 0x55a244513400 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 zone default found 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 4 Realm: () 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 zonegroup default 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513c00 con 0x55a244512400 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66000 con 0x55a244513400 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.469+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66400 con 0x55a244513400 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.470+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.470+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.470+0000 7fc2d2533880 20 started sync module instance, tier type = 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.470+0000 7fc2d2533880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.470+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66800 con 0x55a244512800 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.471+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.471+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94155418487040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66c00 con 0x55a244512800 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.471+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f67000 con 0x55a244f7bc00 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.471+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94155418487040] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.471+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.472+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94155418488192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f67400 con 0x55a244f7bc00 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.472+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f67800 con 0x55a244513400 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.473+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.473+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94155418488192] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.473+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94155418489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66000 con 0x55a244513400 2024-04-27T05:21:34.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.473+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66400 con 0x55a244512800 2024-04-27T05:21:34.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.474+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94155418489344] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.474+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.474+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94155418490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f67800 con 0x55a244512800 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.474+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513800 con 0x55a244f7bc00 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.475+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.475+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94155418491648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244512c00 con 0x55a244f7bc00 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.475+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513000 con 0x55a244512400 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.475+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94155418490496] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.476+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.476+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94155418492800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244513c00 con 0x55a244512400 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.476+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2450dac00 con 0x55a244512400 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.476+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94155418491648] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.477+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94155418492800] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.477+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.477+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94155418493952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66800 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.477+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66c00 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94155418495104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66400 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=0 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=1 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=2 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=3 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=4 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=5 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94155418493952] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.478+0000 7fc2d2533880 20 add_watcher() i=6 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.479+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94155418495104] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.479+0000 7fc2d2533880 20 add_watcher() i=7 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.479+0000 7fc2d2533880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.479+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f67800 con 0x55a244f7bc00 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.479+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.479+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94155418496256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f67000 con 0x55a244f7bc00 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.480+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94155418496256] v29'70 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.480+0000 7fc2b5df0640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.480+0000 7fc2d2533880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.480+0000 7fc2d2533880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.480+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2d2533880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2d2533880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2441a6780 con 0x55a244f67c00 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2ce621640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2ce621640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42770/0 (socket says 172.21.15.138:42770) 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2ce621640 1 -- 172.21.15.138:0/4235714870 learned_addr learned my addr 172.21.15.138:0/4235714870 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2ce621640 1 -- 172.21.15.138:0/4235714870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2441f4680 con 0x55a244f67c00 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.481+0000 7fc2ce621640 1 --2- 172.21.15.138:0/4235714870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55a2451007b0 tx=0x55a2444da0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432a0e746f8ed505 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2b55ef640 1 -- 172.21.15.138:0/4235714870 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a244f68580 con 0x55a244f67c00 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2b55ef640 1 -- 172.21.15.138:0/4235714870 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a245054000 con 0x55a244f67c00 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2b55ef640 1 -- 172.21.15.138:0/4235714870 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a244f686e0 con 0x55a244f67c00 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 -- 172.21.15.138:0/4235714870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 msgr2=0x55a244466680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 --2- 172.21.15.138:0/4235714870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55a2451007b0 tx=0x55a2444da0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 -- 172.21.15.138:0/4235714870 shutdown_connections 2024-04-27T05:21:34.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 --2- 172.21.15.138:0/4235714870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 -- 172.21.15.138:0/4235714870 >> 172.21.15.138:0/4235714870 conn(0x55a244f67400 msgr2=0x55a24428d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 -- 172.21.15.138:0/4235714870 shutdown_connections 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.482+0000 7fc2d2533880 1 -- 172.21.15.138:0/4235714870 wait complete. 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2d2533880 1 Processor -- start 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2d2533880 1 -- start start 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2d2533880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2d2533880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2441a6500 con 0x55a244f67c00 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2ce621640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2ce621640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42772/0 (socket says 172.21.15.138:42772) 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2ce621640 1 -- 172.21.15.138:0/11552214 learned_addr learned my addr 172.21.15.138:0/11552214 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2ce621640 1 -- 172.21.15.138:0/11552214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2441f44e0 con 0x55a244f67c00 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2ce621640 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55a2444f5740 tx=0x55a2444da000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2b3dec640 1 -- 172.21.15.138:0/11552214 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a244f68840 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2b3dec640 1 -- 172.21.15.138:0/11552214 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a245054780 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a244186ea0 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.483+0000 7fc2b3dec640 1 -- 172.21.15.138:0/11552214 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a244f68580 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.484+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2441876c0 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.484+0000 7fc2b3dec640 1 -- 172.21.15.138:0/11552214 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a2441acb00 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.484+0000 7fc2b3dec640 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244f66800 0x55a244467700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.484+0000 7fc2b3dec640 1 -- 172.21.15.138:0/11552214 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a244195dc0 con 0x55a244f67c00 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.484+0000 7fc2d2533880 5 note: GC not initialized 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.484+0000 7fc2cf623640 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244f66800 0x55a244467700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2b15e7640 20 reqs_thread_entry: start 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2cf623640 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244f66800 0x55a244467700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55a245100ba0 tx=0x55a2444da960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2af5e3640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2aedc1640 20 reqs_thread_entry: start 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6000 con 0x55a244f7bc00 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.485+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6400 con 0x55a244512400 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.486+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.486+0000 7fc2d2533880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.486+0000 7fc2d2533880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.486+0000 7fc2d2533880 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.486+0000 7fc2d2533880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.486+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~459 in=459b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6c00 con 0x55a244f7bc00 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~459] v29'4 uv4 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2d2533880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2d2533880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2d2533880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94155418498560 in=647b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f7000 con 0x55a244f7bc00 2024-04-27T05:21:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94155418491648 notify 124554051587 ret 0) v3 ==== 677+0+0 (crc 0 0 0) 0x55a2444d9d40 con 0x55a244f7bc00 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2c9617640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051587 cookie 94155418491648 notifier 4357 bl.length()=635 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2c9617640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2c9617640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(65 notify.4 [notify cookie 94155418498560 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2c9617640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66c00 con 0x55a244f7bc00 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94155418498560 notify 124554051587 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a24457f860 con 0x55a244f7bc00 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.487+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f7400 con 0x55a244512400 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2d2533880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2d2533880 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2d2533880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94155418498560 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f7800 con 0x55a244513400 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94155418489344 notify 124554051584 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55a245098ea0 con 0x55a244513400 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2c9617640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051584 cookie 94155418489344 notifier 4357 bl.length()=177 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2c9617640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2c9617640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2c9617640 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a244f66400 con 0x55a244513400 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [notify cookie 94155418498560 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.489+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94155418498560 notify 124554051584 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a245099040 con 0x55a244513400 2024-04-27T05:21:34.515 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.519 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.490+0000 7fc2b5df0640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.490+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94155418496256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f7c00 con 0x55a244f7bc00 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.491+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94155418496256] v29'71 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.491+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94155418487040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6000 con 0x55a244512800 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.492+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94155418487040] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.492+0000 7fc2d2533880 20 remove_watcher() i=0 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.492+0000 7fc2d2533880 2 removed watcher, disabling cache 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.492+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94155418488192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6400 con 0x55a244f7bc00 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.493+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94155418488192] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.493+0000 7fc2d2533880 20 remove_watcher() i=1 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.493+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94155418489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6c00 con 0x55a244513400 2024-04-27T05:21:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.494+0000 7fc2cee22640 1 -- 172.21.15.138:0/2168965339 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94155418489344] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452afc0 con 0x55a244513400 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.494+0000 7fc2d2533880 20 remove_watcher() i=2 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.494+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94155418490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f6800 con 0x55a244512800 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.495+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94155418490496] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452b200 con 0x55a244512800 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.495+0000 7fc2d2533880 20 remove_watcher() i=3 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.495+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94155418491648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2445f7000 con 0x55a244f7bc00 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.496+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94155418491648] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244f7bc00 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.496+0000 7fc2d2533880 20 remove_watcher() i=4 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.496+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94155418492800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2444ca000 con 0x55a244512400 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.496+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94155418492800] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.497+0000 7fc2d2533880 20 remove_watcher() i=5 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.497+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94155418493952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2444ca400 con 0x55a244512400 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.497+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94155418493952] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.497+0000 7fc2d2533880 20 remove_watcher() i=6 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.497+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94155418495104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a2444ca800 con 0x55a244512400 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.498+0000 7fc2cf623640 1 -- 172.21.15.138:0/2168965339 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94155418495104] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a24452a900 con 0x55a244512400 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.498+0000 7fc2d2533880 20 remove_watcher() i=7 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a244512400 msgr2=0x55a244467180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a244512400 0x55a244467180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a244f7bc00 msgr2=0x55a244465b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a244f7bc00 0x55a244465b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a244512800 msgr2=0x55a244466c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a244512800 0x55a244466c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a244513400 msgr2=0x55a244466100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a244513400 0x55a244466100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244512000 msgr2=0x55a244465600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244512000 0x55a244465600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55a24428ec00 tx=0x55a2444da120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2ce621640 1 -- 172.21.15.138:0/2168965339 reap_dead start 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 msgr2=0x55a244464b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55a24503f950 tx=0x55a244295a40 comp rx=0 tx=0).stop 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 shutdown_connections 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/2168965339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f7b800 0x55a244464b00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 >> 172.21.15.138:0/2168965339 conn(0x55a244183400 msgr2=0x55a24428d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 shutdown_connections 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/2168965339 wait complete. 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244f66800 msgr2=0x55a244467700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244f66800 0x55a244467700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55a245100ba0 tx=0x55a2444da960 comp rx=0 tx=0).stop 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 msgr2=0x55a244466680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55a2444f5740 tx=0x55a2444da000 comp rx=0 tx=0).stop 2024-04-27T05:21:34.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 shutdown_connections 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a244f66800 0x55a244467700 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 --2- 172.21.15.138:0/11552214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244f67c00 0x55a244466680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 >> 172.21.15.138:0/11552214 conn(0x55a244f67400 msgr2=0x55a24428d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 shutdown_connections 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.499+0000 7fc2d2533880 1 -- 172.21.15.138:0/11552214 wait complete. 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a2450da000 msgr2=0x55a244465080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a2450da000 0x55a244465080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55a245084450 tx=0x55a2442954a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 msgr2=0x55a244464580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55a24503eed0 tx=0x55a244294c60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 shutdown_connections 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a2450da000 0x55a244465080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 --2- 172.21.15.138:0/322957503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a244183000 0x55a244464580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 >> 172.21.15.138:0/322957503 conn(0x55a244182c00 msgr2=0x55a24428b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 shutdown_connections 2024-04-27T05:21:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.500+0000 7fc2d2533880 1 -- 172.21.15.138:0/322957503 wait complete. 2024-04-27T05:21:34.530 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.530 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:34.530 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:34.530 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:34.573 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.573 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.571+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.571+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.571+0000 7f76aec84880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.571+0000 7f76aec84880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555f3d7a4600 con 0x555f3d4cd400 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.571+0000 7f76aec84880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x555f3d50c280 con 0x555f3d4cd000 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.572+0000 7f76ad429640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.572+0000 7f76acc28640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x555f3d4cd400 0x555f3d5d4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60792/0 (socket says 172.21.15.138:60792) 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.572+0000 7f76acc28640 1 -- 172.21.15.138:0/3042853900 learned_addr learned my addr 172.21.15.138:0/3042853900 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.572+0000 7f76ad429640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42782/0 (socket says 172.21.15.138:42782) 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.572+0000 7f76ad429640 1 -- 172.21.15.138:0/3042853900 >> v1:172.21.15.114:6789/0 conn(0x555f3d4cd400 legacy=0x555f3d5d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.572+0000 7f76ad429640 1 -- 172.21.15.138:0/3042853900 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d4d1380 con 0x555f3d4cd000 2024-04-27T05:21:34.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.573+0000 7f76ad429640 1 --2- 172.21.15.138:0/3042853900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555f3d5d8d50 tx=0x555f3d5dede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10db1ef0002c1451 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76ac427640 1 -- 172.21.15.138:0/3042853900 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e5ce0 con 0x555f3d4cd000 2024-04-27T05:21:34.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76ac427640 1 -- 172.21.15.138:0/3042853900 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3d7a5b00 con 0x555f3d4cd000 2024-04-27T05:21:34.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76ac427640 1 -- 172.21.15.138:0/3042853900 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e5e40 con 0x555f3d4cd000 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 -- 172.21.15.138:0/3042853900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 msgr2=0x555f3d7ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 --2- 172.21.15.138:0/3042853900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555f3d5d8d50 tx=0x555f3d5dede0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 -- 172.21.15.138:0/3042853900 shutdown_connections 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 --2- 172.21.15.138:0/3042853900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 -- 172.21.15.138:0/3042853900 >> 172.21.15.138:0/3042853900 conn(0x555f3d4ccc00 msgr2=0x555f3d5d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 -- 172.21.15.138:0/3042853900 shutdown_connections 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.574+0000 7f76aec84880 1 -- 172.21.15.138:0/3042853900 wait complete. 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76aec84880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76aec84880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3d50c780 con 0x555f3d4cd000 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76ad429640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76ad429640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42798/0 (socket says 172.21.15.138:42798) 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76ad429640 1 -- 172.21.15.138:0/456718590 learned_addr learned my addr 172.21.15.138:0/456718590 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.575+0000 7f76ad429640 1 -- 172.21.15.138:0/456718590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d4d0ea0 con 0x555f3d4cd000 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76ad429640 1 --2- 172.21.15.138:0/456718590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x555f3e2d1560 tx=0x555f3d5decc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edccebcc99eca88 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76abc26640 1 -- 172.21.15.138:0/456718590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e54a0 con 0x555f3d4cd000 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76abc26640 1 -- 172.21.15.138:0/456718590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3d7a5e00 con 0x555f3d4cd000 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76abc26640 1 -- 172.21.15.138:0/456718590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e5ce0 con 0x555f3d4cd000 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- 172.21.15.138:0/456718590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 msgr2=0x555f3d7ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 --2- 172.21.15.138:0/456718590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x555f3e2d1560 tx=0x555f3d5decc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- 172.21.15.138:0/456718590 shutdown_connections 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 --2- 172.21.15.138:0/456718590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- 172.21.15.138:0/456718590 >> 172.21.15.138:0/456718590 conn(0x555f3d4ccc00 msgr2=0x555f3d5d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- 172.21.15.138:0/456718590 shutdown_connections 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- 172.21.15.138:0/456718590 wait complete. 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.576+0000 7f76aec84880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3d50c8c0 con 0x555f3d4cd000 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76ad429640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76ad429640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42800/0 (socket says 172.21.15.138:42800) 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76ad429640 1 -- 172.21.15.138:0/1130740534 learned_addr learned my addr 172.21.15.138:0/1130740534 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76ad429640 1 -- 172.21.15.138:0/1130740534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d4d16c0 con 0x555f3d4cd000 2024-04-27T05:21:34.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76ad429640 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x555f3e388de0 tx=0x555f3d5dec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76aa423640 1 -- 172.21.15.138:0/1130740534 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e5e40 con 0x555f3d4cd000 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76aa423640 1 -- 172.21.15.138:0/1130740534 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3e2ac000 con 0x555f3d4cd000 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f3d4d11e0 con 0x555f3d4cd000 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76aa423640 1 -- 172.21.15.138:0/1130740534 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e54a0 con 0x555f3d4cd000 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.577+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f3d53eea0 con 0x555f3d4cd000 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aec84880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aec84880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3d50cc80 con 0x555f3e2c4c00 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aa423640 1 -- 172.21.15.138:0/1130740534 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555f3d4f6b00 con 0x555f3d4cd000 2024-04-27T05:21:34.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76adc2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aa423640 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e2c5c00 0x555f3d7af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76adc2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42808/0 (socket says 172.21.15.138:42808) 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76adc2a640 1 -- 172.21.15.138:0/1810937 learned_addr learned my addr 172.21.15.138:0/1810937 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76aa423640 1 -- 172.21.15.138:0/1130740534 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x555f3d4dfa40 con 0x555f3d4cd000 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76ad429640 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e2c5c00 0x555f3d7af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.578+0000 7f76adc2a640 1 -- 172.21.15.138:0/1810937 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d53ed00 con 0x555f3e2c4c00 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76adc2a640 1 --2- 172.21.15.138:0/1810937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x555f3e389b90 tx=0x555f3d5dee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd28914e7020f52 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76ad429640 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e2c5c00 0x555f3d7af080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555f3e3cc2a0 tx=0x555f3d5def60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76a7c1e640 1 -- 172.21.15.138:0/1810937 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e5e40 con 0x555f3e2c4c00 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76a7c1e640 1 -- 172.21.15.138:0/1810937 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3e2ac600 con 0x555f3e2c4c00 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76a7c1e640 1 -- 172.21.15.138:0/1810937 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e54a0 con 0x555f3e2c4c00 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 -- 172.21.15.138:0/1810937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 msgr2=0x555f3d7aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 --2- 172.21.15.138:0/1810937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x555f3e389b90 tx=0x555f3d5dee40 comp rx=0 tx=0).stop 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 -- 172.21.15.138:0/1810937 shutdown_connections 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 --2- 172.21.15.138:0/1810937 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 -- 172.21.15.138:0/1810937 >> 172.21.15.138:0/1810937 conn(0x555f3d4cd400 msgr2=0x555f3d5d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 -- 172.21.15.138:0/1810937 shutdown_connections 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 -- 172.21.15.138:0/1810937 wait complete. 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.579+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76aec84880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76aec84880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3d4f0280 con 0x555f3e2c4c00 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76adc2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76adc2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42822/0 (socket says 172.21.15.138:42822) 2024-04-27T05:21:34.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 learned_addr learned my addr 172.21.15.138:0/231310501 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d53eb60 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76adc2a640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x555f3e389980 tx=0x555f3d5df9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76a641b640 1 -- 172.21.15.138:0/231310501 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e5e40 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76a641b640 1 -- 172.21.15.138:0/231310501 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3e2ac780 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f3d53e9c0 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76a641b640 1 -- 172.21.15.138:0/231310501 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d4e54a0 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.580+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f3d53e820 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76a641b640 1 -- 172.21.15.138:0/231310501 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555f3d4f7600 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76a641b640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3d846000 0x555f3d7af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76a641b640 1 -- 172.21.15.138:0/231310501 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x555f3d4dfc00 con 0x555f3e2c4c00 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76acc28640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3d846000 0x555f3d7af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555f3e2c5000 0x555f3d7afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.581+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e2c5400 con 0x555f3e2c5000 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76adc2a640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555f3e2c5000 0x555f3d7afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76acc28640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3d846000 0x555f3d7af600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x555f3e419260 tx=0x555f3d80c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76adc2a640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555f3e2c5000 0x555f3d7afb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 realm 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e2c5800 con 0x555f3e2c5000 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847000 con 0x555f3e2c5000 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.582+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847400 con 0x555f3e2c5000 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555f3d847800 0x555f3d7b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847c00 con 0x555f3d847800 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76ad429640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555f3d847800 0x555f3d7b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76ad429640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555f3d847800 0x555f3d7b0100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 20 searching for the correct realm 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.583+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847000 con 0x555f3d847800 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.584+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.584+0000 7f76ad429640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555f3d847000 0x555f3d7b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.584+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847c00 con 0x555f3d847000 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.584+0000 7f76acc28640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555f3d847000 0x555f3d7b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.584+0000 7f76acc28640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555f3d847000 0x555f3d7b0c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76acc28640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555f3d847c00 0x555f3d7b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76adc2a640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555f3d847c00 0x555f3d7b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76adc2a640 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555f3d847c00 0x555f3d7b1180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.585+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847000 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.586+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847000 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.587+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.588+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.589+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847000 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.590+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.590+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.590+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.590+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.590+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.590+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847000 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847800 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.591+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847c00 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3e2c5000 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d846c00 con 0x555f3d847000 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76aec84880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76aec84880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76aec84880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76aec84880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.592+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847400 con 0x555f3e2c5000 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e2c5400 con 0x555f3e2c5000 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e2c5800 con 0x555f3d847800 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 zone default found 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 4 Realm: () 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 zonegroup default 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a000 con 0x555f3d847c00 2024-04-27T05:21:34.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.593+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a400 con 0x555f3d847800 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a800 con 0x555f3d847800 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 20 started sync module instance, tier type = 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.594+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86ac00 con 0x555f3d847000 2024-04-27T05:21:34.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.595+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.595+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93867541686528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86b000 con 0x555f3d847000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.595+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86b400 con 0x555f3e2c5000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.596+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93867541686528] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.596+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.596+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93867541687680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86b800 con 0x555f3e2c5000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.596+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86bc00 con 0x555f3d847800 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.597+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.597+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93867541688832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a400 con 0x555f3d847800 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.597+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a800 con 0x555f3d847000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.597+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93867541687680] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.598+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93867541688832] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.598+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.598+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93867541689984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86bc00 con 0x555f3d847000 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.598+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a000 con 0x555f3e2c5000 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.599+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.599+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93867541691136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e2c5800 con 0x555f3e2c5000 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.599+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e2c5400 con 0x555f3d847c00 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.599+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93867541689984] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.600+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.600+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93867541692288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d847400 con 0x555f3d847c00 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.600+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e42b800 con 0x555f3d847c00 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.600+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93867541691136] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.601+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93867541692288] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.601+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.601+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93867541693440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3e42b400 con 0x555f3d847c00 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.601+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86ac00 con 0x555f3d847c00 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93867541693440] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93867541694592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86b000 con 0x555f3d847c00 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=0 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=1 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=2 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=3 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=4 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=5 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.602+0000 7f76aec84880 20 add_watcher() i=6 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.603+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93867541694592] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.603+0000 7f76aec84880 20 add_watcher() i=7 2024-04-27T05:21:34.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.603+0000 7f76aec84880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.603+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86a800 con 0x555f3e2c5000 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.603+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.603+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93867541695744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d86b400 con 0x555f3e2c5000 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.604+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93867541695744] v29'72 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.604+0000 7f76943f7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.604+0000 7f76aec84880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.604+0000 7f76aec84880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.604+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.604+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76aec84880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76aec84880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3d4f08c0 con 0x555f3d846c00 2024-04-27T05:21:34.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76acc28640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76acc28640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42828/0 (socket says 172.21.15.138:42828) 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76acc28640 1 -- 172.21.15.138:0/266264168 learned_addr learned my addr 172.21.15.138:0/266264168 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76acc28640 1 -- 172.21.15.138:0/266264168 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d53e680 con 0x555f3d846c00 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f76acc28640 1 --2- 172.21.15.138:0/266264168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x555f3d828930 tx=0x555f3d80c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173b9d7238d004a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f7693bf6640 1 -- 172.21.15.138:0/266264168 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d86c580 con 0x555f3d846c00 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f7693bf6640 1 -- 172.21.15.138:0/266264168 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3e2ac000 con 0x555f3d846c00 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.605+0000 7f7693bf6640 1 -- 172.21.15.138:0/266264168 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d86c6e0 con 0x555f3d846c00 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 -- 172.21.15.138:0/266264168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 msgr2=0x555f3d7b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 --2- 172.21.15.138:0/266264168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x555f3d828930 tx=0x555f3d80c180 comp rx=0 tx=0).stop 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 -- 172.21.15.138:0/266264168 shutdown_connections 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 --2- 172.21.15.138:0/266264168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 -- 172.21.15.138:0/266264168 >> 172.21.15.138:0/266264168 conn(0x555f3d86b800 msgr2=0x555f3d5d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 -- 172.21.15.138:0/266264168 shutdown_connections 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 -- 172.21.15.138:0/266264168 wait complete. 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 Processor -- start 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.606+0000 7f76aec84880 1 -- start start 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76aec84880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76aec84880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f3d4f0780 con 0x555f3d846c00 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76acc28640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76acc28640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42844/0 (socket says 172.21.15.138:42844) 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76acc28640 1 -- 172.21.15.138:0/3543744365 learned_addr learned my addr 172.21.15.138:0/3543744365 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76acc28640 1 -- 172.21.15.138:0/3543744365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f3d53e4e0 con 0x555f3d846c00 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76acc28640 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x555f3d829860 tx=0x555f3d80cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76923f3640 1 -- 172.21.15.138:0/3543744365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d86c840 con 0x555f3d846c00 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76923f3640 1 -- 172.21.15.138:0/3543744365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555f3e2aca80 con 0x555f3d846c00 2024-04-27T05:21:34.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f3d4d1380 con 0x555f3d846c00 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76923f3640 1 -- 172.21.15.138:0/3543744365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555f3d86c580 con 0x555f3d846c00 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.607+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f3d4d0ea0 con 0x555f3d846c00 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.608+0000 7f76923f3640 1 -- 172.21.15.138:0/3543744365 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555f3d4f7b80 con 0x555f3d846c00 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.608+0000 7f76923f3640 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e42b800 0x555f3d7b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.608+0000 7f76923f3640 1 -- 172.21.15.138:0/3543744365 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x555f3d4dfdc0 con 0x555f3d846c00 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.608+0000 7f76aec84880 5 note: GC not initialized 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.608+0000 7f76adc2a640 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e42b800 0x555f3d7b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.608+0000 7f768fbee640 20 reqs_thread_entry: start 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76aec84880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76adc2a640 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e42b800 0x555f3d7b1700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x555f3e4186c0 tx=0x555f3d80c8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76aec84880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f768dbea640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:04 2024 2024-04-27T05:21:34.630 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f768d3c8640 20 reqs_thread_entry: start 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76aec84880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d940000 con 0x555f3e2c5000 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.609+0000 7f76aec84880 20 rados_obj.operate() r=0 bl.length=459 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.631 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.632 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.633 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.634 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.610+0000 7f76943f7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.610+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93867541695744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d940400 con 0x555f3e2c5000 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.611+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93867541695744] v29'73 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.611+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93867541686528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d940800 con 0x555f3d847000 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.612+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93867541686528] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.612+0000 7f76aec84880 20 remove_watcher() i=0 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.612+0000 7f76aec84880 2 removed watcher, disabling cache 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.612+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93867541687680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d940c00 con 0x555f3e2c5000 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.613+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93867541687680] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.613+0000 7f76aec84880 20 remove_watcher() i=1 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.613+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93867541688832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d941000 con 0x555f3d847800 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.614+0000 7f76ad429640 1 -- 172.21.15.138:0/231310501 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93867541688832] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85cfc0 con 0x555f3d847800 2024-04-27T05:21:34.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.614+0000 7f76aec84880 20 remove_watcher() i=2 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.614+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93867541689984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d941400 con 0x555f3d847000 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.615+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93867541689984] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85d200 con 0x555f3d847000 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.615+0000 7f76aec84880 20 remove_watcher() i=3 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.615+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93867541691136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d941800 con 0x555f3e2c5000 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.616+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93867541691136] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3e2c5000 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.616+0000 7f76aec84880 20 remove_watcher() i=4 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.616+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93867541692288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d941c00 con 0x555f3d847c00 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.617+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93867541692288] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.617+0000 7f76aec84880 20 remove_watcher() i=5 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.617+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93867541693440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d940000 con 0x555f3d847c00 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.618+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93867541693440] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.618+0000 7f76aec84880 20 remove_watcher() i=6 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.618+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93867541694592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f3d940400 con 0x555f3d847c00 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76adc2a640 1 -- 172.21.15.138:0/231310501 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93867541694592] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555f3d85c900 con 0x555f3d847c00 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 20 remove_watcher() i=7 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555f3d847c00 msgr2=0x555f3d7b1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555f3d847c00 0x555f3d7b1180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555f3e2c5000 msgr2=0x555f3d7afb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555f3e2c5000 0x555f3d7afb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555f3d847000 msgr2=0x555f3d7b0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555f3d847000 0x555f3d7b0c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555f3d847800 msgr2=0x555f3d7b0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555f3d847800 0x555f3d7b0100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3d846000 msgr2=0x555f3d7af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3d846000 0x555f3d7af600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x555f3e419260 tx=0x555f3d80c120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 msgr2=0x555f3d7aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 --2- 172.21.15.138:0/231310501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3e2c4c00 0x555f3d7aeb00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x555f3e389980 tx=0x555f3d5df9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 reap_dead start 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76acc28640 1 -- 172.21.15.138:0/231310501 reap_dead start 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 shutdown_connections 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 >> 172.21.15.138:0/231310501 conn(0x555f3d4cd400 msgr2=0x555f3d5d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 shutdown_connections 2024-04-27T05:21:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.619+0000 7f76aec84880 1 -- 172.21.15.138:0/231310501 wait complete. 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e42b800 msgr2=0x555f3d7b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e42b800 0x555f3d7b1700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x555f3e4186c0 tx=0x555f3d80c8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 msgr2=0x555f3d7b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x555f3d829860 tx=0x555f3d80cae0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 shutdown_connections 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e42b800 0x555f3d7b1700 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/3543744365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d846c00 0x555f3d7b0680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 >> 172.21.15.138:0/3543744365 conn(0x555f3d86b800 msgr2=0x555f3d5d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 shutdown_connections 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/3543744365 wait complete. 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e2c5c00 msgr2=0x555f3d7af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e2c5c00 0x555f3d7af080 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555f3e3cc2a0 tx=0x555f3d5def60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 msgr2=0x555f3d7ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x555f3e388de0 tx=0x555f3d5dec60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 shutdown_connections 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555f3e2c5c00 0x555f3d7af080 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 --2- 172.21.15.138:0/1130740534 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555f3d4cd000 0x555f3d7ae580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 >> 172.21.15.138:0/1130740534 conn(0x555f3d4ccc00 msgr2=0x555f3d5d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 shutdown_connections 2024-04-27T05:21:34.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.620+0000 7f76aec84880 1 -- 172.21.15.138:0/1130740534 wait complete. 2024-04-27T05:21:34.647 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.648 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-27T05:21:34.648 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-27T05:21:34.648 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-04-27T05:21:34.725 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.725 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.723+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e44013880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a8e37b3400 0x55a8e3a94580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e44013880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a8e3a8a600 con 0x55a8e37b3000 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e44013880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a8e37f2280 con 0x55a8e37b3400 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e3fd71640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a8e37b3400 0x55a8e3a94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e3fd71640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a8e37b3400 0x55a8e3a94580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42854/0 (socket says 172.21.15.138:42854) 2024-04-27T05:21:34.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e40572640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a8e37b3000 0x55a8e38ba800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60796/0 (socket says 172.21.15.138:60796) 2024-04-27T05:21:34.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e3fd71640 1 -- 172.21.15.138:0/2499889506 learned_addr learned my addr 172.21.15.138:0/2499889506 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e3fd71640 1 -- 172.21.15.138:0/2499889506 >> v1:172.21.15.114:6789/0 conn(0x55a8e37b3000 legacy=0x55a8e38ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.724+0000 7f8e3fd71640 1 -- 172.21.15.138:0/2499889506 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e37b7380 con 0x55a8e37b3400 2024-04-27T05:21:34.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.726+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/2499889506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3400 0x55a8e3a94580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55a8e38bec00 tx=0x55a8e38c4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5170086e5eb45593 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.726+0000 7f8e3f570640 1 -- 172.21.15.138:0/2499889506 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cbce0 con 0x55a8e37b3400 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.726+0000 7f8e3f570640 1 -- 172.21.15.138:0/2499889506 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e3a8bb00 con 0x55a8e37b3400 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.726+0000 7f8e3f570640 1 -- 172.21.15.138:0/2499889506 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cbe40 con 0x55a8e37b3400 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.726+0000 7f8e44013880 1 -- 172.21.15.138:0/2499889506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3400 msgr2=0x55a8e3a94580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.726+0000 7f8e44013880 1 --2- 172.21.15.138:0/2499889506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3400 0x55a8e3a94580 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55a8e38bec00 tx=0x55a8e38c4de0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 -- 172.21.15.138:0/2499889506 shutdown_connections 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 --2- 172.21.15.138:0/2499889506 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3400 0x55a8e3a94580 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 -- 172.21.15.138:0/2499889506 >> 172.21.15.138:0/2499889506 conn(0x55a8e37b2c00 msgr2=0x55a8e38ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 -- 172.21.15.138:0/2499889506 shutdown_connections 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 -- 172.21.15.138:0/2499889506 wait complete. 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.727+0000 7f8e44013880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e37f2780 con 0x55a8e37b3000 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e40572640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e40572640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42856/0 (socket says 172.21.15.138:42856) 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e40572640 1 -- 172.21.15.138:0/117334090 learned_addr learned my addr 172.21.15.138:0/117334090 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e40572640 1 -- 172.21.15.138:0/117334090 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e37b6ea0 con 0x55a8e37b3000 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e40572640 1 --2- 172.21.15.138:0/117334090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55a8e466f290 tx=0x55a8e38c4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29c00a9b5725604 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e3ed6f640 1 -- 172.21.15.138:0/117334090 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cb4a0 con 0x55a8e37b3000 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e3ed6f640 1 -- 172.21.15.138:0/117334090 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e3a8be00 con 0x55a8e37b3000 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.728+0000 7f8e3ed6f640 1 -- 172.21.15.138:0/117334090 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cbce0 con 0x55a8e37b3000 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- 172.21.15.138:0/117334090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 msgr2=0x55a8e3a94580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 --2- 172.21.15.138:0/117334090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55a8e466f290 tx=0x55a8e38c4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- 172.21.15.138:0/117334090 shutdown_connections 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 --2- 172.21.15.138:0/117334090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- 172.21.15.138:0/117334090 >> 172.21.15.138:0/117334090 conn(0x55a8e37b2c00 msgr2=0x55a8e38bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- 172.21.15.138:0/117334090 shutdown_connections 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- 172.21.15.138:0/117334090 wait complete. 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e44013880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e37f28c0 con 0x55a8e37b3000 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e40572640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e40572640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42866/0 (socket says 172.21.15.138:42866) 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.729+0000 7f8e40572640 1 -- 172.21.15.138:0/2315981939 learned_addr learned my addr 172.21.15.138:0/2315981939 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e40572640 1 -- 172.21.15.138:0/2315981939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e37b76c0 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e40572640 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55a8e466eb40 tx=0x55a8e38c4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e3d56c640 1 -- 172.21.15.138:0/2315981939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cbe40 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e3d56c640 1 -- 172.21.15.138:0/2315981939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e4684000 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8e37b71e0 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e3d56c640 1 -- 172.21.15.138:0/2315981939 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cb4a0 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.730+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8e3824ea0 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e3d56c640 1 -- 172.21.15.138:0/2315981939 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a8e37dd600 con 0x55a8e37b3000 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e44013880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e44013880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e37d6280 con 0x55a8e45ab800 2024-04-27T05:21:34.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e3d56c640 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e4704000 0x55a8e3a95080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e3d56c640 1 -- 172.21.15.138:0/2315981939 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a8e37c5a40 con 0x55a8e37b3000 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40d73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40572640 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e4704000 0x55a8e3a95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40d73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42870/0 (socket says 172.21.15.138:42870) 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40d73640 1 -- 172.21.15.138:0/2066836286 learned_addr learned my addr 172.21.15.138:0/2066836286 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40d73640 1 -- 172.21.15.138:0/2066836286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e3824d00 con 0x55a8e45ab800 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40572640 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e4704000 0x55a8e3a95080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55a8e46b4420 tx=0x55a8e38c54a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e40d73640 1 --2- 172.21.15.138:0/2066836286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55a8e466f530 tx=0x55a8e38c4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac230691ae7f1851 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e3ad67640 1 -- 172.21.15.138:0/2066836286 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cbe40 con 0x55a8e45ab800 2024-04-27T05:21:34.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e3ad67640 1 -- 172.21.15.138:0/2066836286 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e4684300 con 0x55a8e45ab800 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.731+0000 7f8e3ad67640 1 -- 172.21.15.138:0/2066836286 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cb4a0 con 0x55a8e45ab800 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- 172.21.15.138:0/2066836286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 msgr2=0x55a8e3a94b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 --2- 172.21.15.138:0/2066836286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55a8e466f530 tx=0x55a8e38c4c00 comp rx=0 tx=0).stop 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- 172.21.15.138:0/2066836286 shutdown_connections 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 --2- 172.21.15.138:0/2066836286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- 172.21.15.138:0/2066836286 >> 172.21.15.138:0/2066836286 conn(0x55a8e37b3400 msgr2=0x55a8e38bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- 172.21.15.138:0/2066836286 shutdown_connections 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- 172.21.15.138:0/2066836286 wait complete. 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e44013880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e37d7180 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e40d73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e40d73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42880/0 (socket says 172.21.15.138:42880) 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.732+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 learned_addr learned my addr 172.21.15.138:0/417512184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e3824b60 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e40d73640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55a8e466f920 tx=0x55a8e38c5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e39564640 1 -- 172.21.15.138:0/417512184 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cbe40 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e39564640 1 -- 172.21.15.138:0/417512184 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e4684480 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8e38249c0 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e39564640 1 -- 172.21.15.138:0/417512184 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e37cb4a0 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.733+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8e3824820 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e39564640 1 -- 172.21.15.138:0/417512184 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a8e37dd8c0 con 0x55a8e45ab800 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e39564640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b3e000 0x55a8e3a95600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e39564640 1 -- 172.21.15.138:0/417512184 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a8e37c5c00 con 0x55a8e45ab800 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b3e000 0x55a8e3a95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a8e45abc00 0x55a8e3a95b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3e400 con 0x55a8e45abc00 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e40d73640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a8e45abc00 0x55a8e3a95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b3e000 0x55a8e3a95600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55a8e38bec00 tx=0x55a8e3b06120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.734+0000 7f8e40d73640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a8e45abc00 0x55a8e3a95b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 realm 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3e800 con 0x55a8e45abc00 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3ec00 con 0x55a8e45abc00 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3f000 con 0x55a8e45abc00 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a8e3b3f400 0x55a8e3a96100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3e400 con 0x55a8e3b3f400 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.735+0000 7f8e40572640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a8e3b3f400 0x55a8e3a96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e40572640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a8e3b3f400 0x55a8e3a96100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e44013880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e44013880 20 searching for the correct realm 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3e800 con 0x55a8e3b3f400 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e40572640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a8e3b3e800 0x55a8e3a96c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3e400 con 0x55a8e3b3e800 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.736+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a8e3b3e800 0x55a8e3a96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.737+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a8e3b3e800 0x55a8e3a96c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.737+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.737+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a8e3b3e400 0x55a8e3a97180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.737+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.737+0000 7f8e40d73640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a8e3b3e400 0x55a8e3a97180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.737+0000 7f8e40d73640 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a8e3b3e400 0x55a8e3a97180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.738+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e800 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.739+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e800 2024-04-27T05:21:34.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.740+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.740+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.740+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.740+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.740+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.740+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.741+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e800 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.742+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e800 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.743+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3f400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e45abc00 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e45aa800 con 0x55a8e3b3e800 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e44013880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e44013880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e44013880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e44013880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.744+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3ec00 con 0x55a8e45abc00 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3f000 con 0x55a8e45abc00 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3f800 con 0x55a8e3b3f400 2024-04-27T05:21:34.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 zone default found 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 4 Realm: () 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 zonegroup default 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3fc00 con 0x55a8e3b3e400 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.745+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e000 con 0x55a8e3b3f400 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e400 con 0x55a8e3b3f400 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 20 started sync module instance, tier type = 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.746+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e800 con 0x55a8e3b3e800 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.747+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.747+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94183862356224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5ec00 con 0x55a8e3b3e800 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.747+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5f000 con 0x55a8e45abc00 2024-04-27T05:21:34.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.748+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94183862356224] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.748+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.748+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94183862357376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5f400 con 0x55a8e45abc00 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.748+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5f800 con 0x55a8e3b3f400 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.749+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.749+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94183862357376] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.749+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94183862358528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e000 con 0x55a8e3b3f400 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.749+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e400 con 0x55a8e3b3e800 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.750+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94183862358528] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.750+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.750+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94183862359680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5f800 con 0x55a8e3b3e800 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.750+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3f800 con 0x55a8e45abc00 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.751+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.751+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94183862360832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3ec00 con 0x55a8e45abc00 2024-04-27T05:21:34.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.751+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3f000 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.751+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94183862359680] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.752+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.752+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94183862361984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b3fc00 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.752+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e4705000 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.752+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94183862360832] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.753+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.753+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94183862363136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e800 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.753+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5ec00 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.753+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94183862361984] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94183862363136] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94183862364288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5e400 con 0x55a8e3b3e400 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=0 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=1 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=2 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=3 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=4 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=5 2024-04-27T05:21:34.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=6 2024-04-27T05:21:34.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94183862364288] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 20 add_watcher() i=7 2024-04-27T05:21:34.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.754+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5f800 con 0x55a8e45abc00 2024-04-27T05:21:34.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.755+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.755+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94183862365440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5f000 con 0x55a8e45abc00 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94183862365440] v29'74 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e27540640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e44013880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e44013880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e44013880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e44013880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e37d6780 con 0x55a8e3b5fc00 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e3fd71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e3fd71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42888/0 (socket says 172.21.15.138:42888) 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.756+0000 7f8e3fd71640 1 -- 172.21.15.138:0/130343980 learned_addr learned my addr 172.21.15.138:0/130343980 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.757+0000 7f8e3fd71640 1 -- 172.21.15.138:0/130343980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e3824680 con 0x55a8e3b5fc00 2024-04-27T05:21:34.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.757+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/130343980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55a8e47367b0 tx=0x55a8e3b060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992e432a73c09274 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.757+0000 7f8e26d3f640 1 -- 172.21.15.138:0/130343980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e3b60580 con 0x55a8e3b5fc00 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.757+0000 7f8e26d3f640 1 -- 172.21.15.138:0/130343980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e4684000 con 0x55a8e3b5fc00 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.757+0000 7f8e26d3f640 1 -- 172.21.15.138:0/130343980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e3b606e0 con 0x55a8e3b5fc00 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- 172.21.15.138:0/130343980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 msgr2=0x55a8e3a96680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 --2- 172.21.15.138:0/130343980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55a8e47367b0 tx=0x55a8e3b060c0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- 172.21.15.138:0/130343980 shutdown_connections 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 --2- 172.21.15.138:0/130343980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- 172.21.15.138:0/130343980 >> 172.21.15.138:0/130343980 conn(0x55a8e3b5f400 msgr2=0x55a8e38bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- 172.21.15.138:0/130343980 shutdown_connections 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- 172.21.15.138:0/130343980 wait complete. 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 Processor -- start 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- start start 2024-04-27T05:21:34.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e44013880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8e37d6500 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e3fd71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e3fd71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42896/0 (socket says 172.21.15.138:42896) 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.758+0000 7f8e3fd71640 1 -- 172.21.15.138:0/3157769564 learned_addr learned my addr 172.21.15.138:0/3157769564 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e3fd71640 1 -- 172.21.15.138:0/3157769564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8e38244e0 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e3fd71640 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55a8e3b21620 tx=0x55a8e3b06000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e2553c640 1 -- 172.21.15.138:0/3157769564 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e3b60840 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e2553c640 1 -- 172.21.15.138:0/3157769564 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8e4684780 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e2553c640 1 -- 172.21.15.138:0/3157769564 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a8e3b60580 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a8e37b6ea0 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.759+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8e37b76c0 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e2553c640 1 -- 172.21.15.138:0/3157769564 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a8e37dcb00 con 0x55a8e3b5fc00 2024-04-27T05:21:34.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e2553c640 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b5e800 0x55a8e3a97700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e2553c640 1 -- 172.21.15.138:0/3157769564 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a8e37c5dc0 con 0x55a8e3b5fc00 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e44013880 5 note: GC not initialized 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e40d73640 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b5e800 0x55a8e3a97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e22d37640 20 reqs_thread_entry: start 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e44013880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e40d73640 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b5e800 0x55a8e3a97700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55a8e3b20db0 tx=0x55a8e3b06960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e44013880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e20d33640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e20511640 20 reqs_thread_entry: start 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e44013880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e44013880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.760+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c24000 con 0x55a8e45abc00 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.761+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.761+0000 7f8e44013880 20 rados_obj.operate() r=0 bl.length=459 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.761+0000 7f8e44013880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.761+0000 7f8e44013880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.761+0000 7f8e44013880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.761+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c24800 con 0x55a8e45abc00 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v29'5 uv5 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e44013880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e44013880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e44013880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94183862367744 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c24c00 con 0x55a8e45abc00 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94183862360832 notify 124554051588 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55a8e3b05d40 con 0x55a8e45abc00 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e3ad67640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051588 cookie 94183862360832 notifier 4373 bl.length()=533 2024-04-27T05:21:34.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e3ad67640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e3ad67640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 94183862367744 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e3ad67640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c38000 con 0x55a8e45abc00 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.762+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.763+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94183862367744 notify 124554051588 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a8e3bab860 con 0x55a8e45abc00 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.763+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c25000 con 0x55a8e3b3e400 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v29'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e44013880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94183862367744 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c25400 con 0x55a8e3b3f400 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94183862358528 notify 124554051585 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55a8e46c8ea0 con 0x55a8e3b3f400 2024-04-27T05:21:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e3a566640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051585 cookie 94183862358528 notifier 4373 bl.length()=170 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(67 notify.2 [notify cookie 94183862367744 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e3a566640 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3b5ec00 con 0x55a8e3b3f400 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.764+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94183862367744 notify 124554051585 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a8e46c9040 con 0x55a8e3b3f400 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:34.792 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.793 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.795 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.765+0000 7f8e27540640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.765+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94183862365440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c25800 con 0x55a8e45abc00 2024-04-27T05:21:34.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.766+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94183862365440] v29'75 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.766+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94183862356224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c25c00 con 0x55a8e3b3e800 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.767+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94183862356224] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.767+0000 7f8e44013880 20 remove_watcher() i=0 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.767+0000 7f8e44013880 2 removed watcher, disabling cache 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.767+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94183862357376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c38000 con 0x55a8e45abc00 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.768+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94183862357376] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.768+0000 7f8e44013880 20 remove_watcher() i=1 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.768+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94183862358528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c38800 con 0x55a8e3b3f400 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.769+0000 7f8e40572640 1 -- 172.21.15.138:0/417512184 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94183862358528] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54d80 con 0x55a8e3b3f400 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.769+0000 7f8e44013880 20 remove_watcher() i=2 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.769+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94183862359680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c38c00 con 0x55a8e3b3e800 2024-04-27T05:21:34.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.770+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94183862359680] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b54fc0 con 0x55a8e3b3e800 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.770+0000 7f8e44013880 20 remove_watcher() i=3 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.770+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94183862360832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c39000 con 0x55a8e45abc00 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.771+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94183862360832] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e45abc00 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.771+0000 7f8e44013880 20 remove_watcher() i=4 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.771+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94183862361984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c39400 con 0x55a8e3b3e400 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.772+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94183862361984] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.772+0000 7f8e44013880 20 remove_watcher() i=5 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.772+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94183862363136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c39800 con 0x55a8e3b3e400 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.773+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94183862363136] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.773+0000 7f8e44013880 20 remove_watcher() i=6 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.773+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94183862364288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a8e3c24000 con 0x55a8e3b3e400 2024-04-27T05:21:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e40d73640 1 -- 172.21.15.138:0/417512184 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94183862364288] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a8e3b546c0 con 0x55a8e3b3e400 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 20 remove_watcher() i=7 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a8e3b3e400 msgr2=0x55a8e3a97180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a8e3b3e400 0x55a8e3a97180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a8e45abc00 msgr2=0x55a8e3a95b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a8e45abc00 0x55a8e3a95b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a8e3b3e800 msgr2=0x55a8e3a96c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a8e3b3e800 0x55a8e3a96c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a8e3b3f400 msgr2=0x55a8e3a96100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a8e3b3f400 0x55a8e3a96100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b3e000 msgr2=0x55a8e3a95600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b3e000 0x55a8e3a95600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55a8e38bec00 tx=0x55a8e3b06120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 msgr2=0x55a8e3a94b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55a8e466f920 tx=0x55a8e38c5a40 comp rx=0 tx=0).stop 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e3fd71640 1 -- 172.21.15.138:0/417512184 reap_dead start 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 shutdown_connections 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 --2- 172.21.15.138:0/417512184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e45ab800 0x55a8e3a94b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 >> 172.21.15.138:0/417512184 conn(0x55a8e37b3400 msgr2=0x55a8e38bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 shutdown_connections 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.774+0000 7f8e44013880 1 -- 172.21.15.138:0/417512184 wait complete. 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b5e800 msgr2=0x55a8e3a97700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b5e800 0x55a8e3a97700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55a8e3b20db0 tx=0x55a8e3b06960 comp rx=0 tx=0).stop 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 msgr2=0x55a8e3a96680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55a8e3b21620 tx=0x55a8e3b06000 comp rx=0 tx=0).stop 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 shutdown_connections 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e3b5e800 0x55a8e3a97700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/3157769564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e3b5fc00 0x55a8e3a96680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 >> 172.21.15.138:0/3157769564 conn(0x55a8e3b5f400 msgr2=0x55a8e38bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 shutdown_connections 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/3157769564 wait complete. 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e4704000 msgr2=0x55a8e3a95080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e4704000 0x55a8e3a95080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55a8e46b4420 tx=0x55a8e38c54a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 msgr2=0x55a8e3a94580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55a8e466eb40 tx=0x55a8e38c4c60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 shutdown_connections 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a8e4704000 0x55a8e3a95080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 --2- 172.21.15.138:0/2315981939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8e37b3000 0x55a8e3a94580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 >> 172.21.15.138:0/2315981939 conn(0x55a8e37b2c00 msgr2=0x55a8e38bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 shutdown_connections 2024-04-27T05:21:34.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.775+0000 7f8e44013880 1 -- 172.21.15.138:0/2315981939 wait complete. 2024-04-27T05:21:34.807 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.807 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-27T05:21:34.807 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-27T05:21:34.807 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-04-27T05:21:34.852 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.852 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14ccc0f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fcd3861400 0x55fcd3b42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14ccc0f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fcd3b38600 con 0x55fcd3861000 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14ccc0f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fcd38a0280 con 0x55fcd3861400 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14c884c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fcd3861400 0x55fcd3b42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14c904d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55fcd3861000 0x55fcd3968800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60812/0 (socket says 172.21.15.138:60812) 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14c904d640 1 -- 172.21.15.138:0/1785840693 learned_addr learned my addr 172.21.15.138:0/1785840693 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14c884c640 1 -- 172.21.15.138:0/1785840693 >> v1:172.21.15.114:6789/0 conn(0x55fcd3861000 legacy=0x55fcd3968800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.851+0000 7f14c884c640 1 -- 172.21.15.138:0/1785840693 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd3865380 con 0x55fcd3861400 2024-04-27T05:21:34.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.853+0000 7f14c884c640 1 --2- 172.21.15.138:0/1785840693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861400 0x55fcd3b42580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55fcd396cc00 tx=0x55fcd3972de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c726066208c1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.853+0000 7f14c804b640 1 -- 172.21.15.138:0/1785840693 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd3879ce0 con 0x55fcd3861400 2024-04-27T05:21:34.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.853+0000 7f14c804b640 1 -- 172.21.15.138:0/1785840693 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd3b39b00 con 0x55fcd3861400 2024-04-27T05:21:34.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.853+0000 7f14c804b640 1 -- 172.21.15.138:0/1785840693 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd3879e40 con 0x55fcd3861400 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- 172.21.15.138:0/1785840693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861400 msgr2=0x55fcd3b42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/1785840693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861400 0x55fcd3b42580 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55fcd396cc00 tx=0x55fcd3972de0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- 172.21.15.138:0/1785840693 shutdown_connections 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/1785840693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861400 0x55fcd3b42580 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- 172.21.15.138:0/1785840693 >> 172.21.15.138:0/1785840693 conn(0x55fcd3860c00 msgr2=0x55fcd3968000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- 172.21.15.138:0/1785840693 shutdown_connections 2024-04-27T05:21:34.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- 172.21.15.138:0/1785840693 wait complete. 2024-04-27T05:21:34.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.854+0000 7f14ccc0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd38a0780 con 0x55fcd3861000 2024-04-27T05:21:34.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c904d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c904d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42920/0 (socket says 172.21.15.138:42920) 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c904d640 1 -- 172.21.15.138:0/3532163739 learned_addr learned my addr 172.21.15.138:0/3532163739 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c904d640 1 -- 172.21.15.138:0/3532163739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd3864ea0 con 0x55fcd3861000 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c904d640 1 --2- 172.21.15.138:0/3532163739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55fcd471ccc0 tx=0x55fcd3972cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6171b7455bd0316 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c784a640 1 -- 172.21.15.138:0/3532163739 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd38794a0 con 0x55fcd3861000 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c784a640 1 -- 172.21.15.138:0/3532163739 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd3b39e00 con 0x55fcd3861000 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.855+0000 7f14c784a640 1 -- 172.21.15.138:0/3532163739 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd3879ce0 con 0x55fcd3861000 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- 172.21.15.138:0/3532163739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 msgr2=0x55fcd3b42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/3532163739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55fcd471ccc0 tx=0x55fcd3972cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- 172.21.15.138:0/3532163739 shutdown_connections 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/3532163739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- 172.21.15.138:0/3532163739 >> 172.21.15.138:0/3532163739 conn(0x55fcd3860c00 msgr2=0x55fcd3969000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- 172.21.15.138:0/3532163739 shutdown_connections 2024-04-27T05:21:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- 172.21.15.138:0/3532163739 wait complete. 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14ccc0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd38a08c0 con 0x55fcd3861000 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14c904d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14c904d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42936/0 (socket says 172.21.15.138:42936) 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.856+0000 7f14c904d640 1 -- 172.21.15.138:0/2116225708 learned_addr learned my addr 172.21.15.138:0/2116225708 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14c904d640 1 -- 172.21.15.138:0/2116225708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd38656c0 con 0x55fcd3861000 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14c904d640 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55fcd471cea0 tx=0x55fcd3972c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14c6047640 1 -- 172.21.15.138:0/2116225708 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd3879e40 con 0x55fcd3861000 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14c6047640 1 -- 172.21.15.138:0/2116225708 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd4732000 con 0x55fcd3861000 2024-04-27T05:21:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fcd38651e0 con 0x55fcd3861000 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14c6047640 1 -- 172.21.15.138:0/2116225708 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd38794a0 con 0x55fcd3861000 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcd38d2ea0 con 0x55fcd3861000 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.857+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c6047640 1 -- 172.21.15.138:0/2116225708 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fcd388b600 con 0x55fcd3861000 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14ccc0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14ccc0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd3884280 con 0x55fcd4659800 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c6047640 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd47b6000 0x55fcd3b43080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c6047640 1 -- 172.21.15.138:0/2116225708 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55fcd3873a40 con 0x55fcd3861000 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c984e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c984e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42938/0 (socket says 172.21.15.138:42938) 2024-04-27T05:21:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c984e640 1 -- 172.21.15.138:0/992999104 learned_addr learned my addr 172.21.15.138:0/992999104 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c904d640 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd47b6000 0x55fcd3b43080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c984e640 1 -- 172.21.15.138:0/992999104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd38d2d00 con 0x55fcd4659800 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c904d640 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd47b6000 0x55fcd3b43080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55fcd4766450 tx=0x55fcd39734a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c984e640 1 --2- 172.21.15.138:0/992999104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55fcd471df50 tx=0x55fcd3972c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab87c0740ce2b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c3842640 1 -- 172.21.15.138:0/992999104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd3879e40 con 0x55fcd4659800 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c3842640 1 -- 172.21.15.138:0/992999104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd4732300 con 0x55fcd4659800 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.858+0000 7f14c3842640 1 -- 172.21.15.138:0/992999104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd38794a0 con 0x55fcd4659800 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- 172.21.15.138:0/992999104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 msgr2=0x55fcd3b42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/992999104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55fcd471df50 tx=0x55fcd3972c00 comp rx=0 tx=0).stop 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- 172.21.15.138:0/992999104 shutdown_connections 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/992999104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- 172.21.15.138:0/992999104 >> 172.21.15.138:0/992999104 conn(0x55fcd3861400 msgr2=0x55fcd396b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- 172.21.15.138:0/992999104 shutdown_connections 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- 172.21.15.138:0/992999104 wait complete. 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14ccc0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd3885180 con 0x55fcd4659800 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14c984e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14c984e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42948/0 (socket says 172.21.15.138:42948) 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.859+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 learned_addr learned my addr 172.21.15.138:0/4149148856 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd38d2b60 con 0x55fcd4659800 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14c984e640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55fcd4665470 tx=0x55fcd39731a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14c203f640 1 -- 172.21.15.138:0/4149148856 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd3879e40 con 0x55fcd4659800 2024-04-27T05:21:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14c203f640 1 -- 172.21.15.138:0/4149148856 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd4732480 con 0x55fcd4659800 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fcd38d29c0 con 0x55fcd4659800 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14c203f640 1 -- 172.21.15.138:0/4149148856 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd38794a0 con 0x55fcd4659800 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.860+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcd38d2820 con 0x55fcd4659800 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14c203f640 1 -- 172.21.15.138:0/4149148856 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fcd388b8c0 con 0x55fcd4659800 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14c203f640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd3be4000 0x55fcd3b43600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14c203f640 1 -- 172.21.15.138:0/4149148856 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55fcd3873c00 con 0x55fcd4659800 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14c884c640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd3be4000 0x55fcd3b43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fcd4659c00 0x55fcd3b43b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4400 con 0x55fcd4659c00 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14c984e640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fcd4659c00 0x55fcd3b43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.861+0000 7f14c884c640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd3be4000 0x55fcd3b43600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55fcd396cc00 tx=0x55fcd3bac120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14c984e640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fcd4659c00 0x55fcd3b43b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 realm 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4800 con 0x55fcd4659c00 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4c00 con 0x55fcd4659c00 2024-04-27T05:21:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5000 con 0x55fcd4659c00 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fcd3be5400 0x55fcd3b44100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.862+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4400 con 0x55fcd3be5400 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14c904d640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fcd3be5400 0x55fcd3b44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14c904d640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fcd3be5400 0x55fcd3b44100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14ccc0f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14ccc0f880 20 searching for the correct realm 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4800 con 0x55fcd3be5400 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.863+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c904d640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fcd3be4800 0x55fcd3b44c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4400 con 0x55fcd3be4800 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c884c640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fcd3be4800 0x55fcd3b44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c884c640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fcd3be4800 0x55fcd3b44c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c884c640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fcd3be4400 0x55fcd3b45180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.864+0000 7f14c984e640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fcd3be4400 0x55fcd3b45180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fcd3be4400 0x55fcd3b45180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.865+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4800 2024-04-27T05:21:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.866+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4800 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.867+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.868+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.869+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4800 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.870+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4800 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be5400 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4400 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.871+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd4659c00 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4658800 con 0x55fcd3be4800 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4c00 con 0x55fcd4659c00 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5000 con 0x55fcd4659c00 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.872+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5800 con 0x55fcd3be5400 2024-04-27T05:21:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 zone default found 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 4 Realm: () 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 zonegroup default 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5c00 con 0x55fcd3be4400 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644000 con 0x55fcd3be5400 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644400 con 0x55fcd3be5400 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 started sync module instance, tier type = 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.873+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644800 con 0x55fcd3be4800 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.874+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.874+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94544371886336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644c00 con 0x55fcd3be4800 2024-04-27T05:21:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.874+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4645000 con 0x55fcd4659c00 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.875+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94544371886336] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.876+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.876+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94544371887488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4645400 con 0x55fcd4659c00 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.876+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4645800 con 0x55fcd3be5400 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.876+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.876+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94544371888640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644000 con 0x55fcd3be5400 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.876+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644400 con 0x55fcd3be4800 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.877+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94544371887488] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.877+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.877+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94544371888640] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.877+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94544371889792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4645800 con 0x55fcd3be4800 2024-04-27T05:21:34.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.877+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5800 con 0x55fcd4659c00 2024-04-27T05:21:34.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.878+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.878+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94544371890944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be4c00 con 0x55fcd4659c00 2024-04-27T05:21:34.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.878+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5000 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.878+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94544371889792] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.879+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.879+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94544371890944] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.879+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94544371892096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3be5c00 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.879+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd47b7800 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.880+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94544371892096] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.880+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.880+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94544371893248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644800 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.880+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644c00 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94544371893248] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94544371894400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644400 con 0x55fcd3be4400 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=0 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=1 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=2 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=3 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=4 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=5 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.881+0000 7f14ccc0f880 20 add_watcher() i=6 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.882+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94544371894400] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.882+0000 7f14ccc0f880 20 add_watcher() i=7 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.882+0000 7f14ccc0f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.882+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4645800 con 0x55fcd4659c00 2024-04-27T05:21:34.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.883+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.883+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94544371895552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4645000 con 0x55fcd4659c00 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94544371895552] v29'76 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14b001b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14ccc0f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14ccc0f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14ccc0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14ccc0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd3884780 con 0x55fcd4645c00 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14c884c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14c884c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42962/0 (socket says 172.21.15.138:42962) 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.884+0000 7f14c884c640 1 -- 172.21.15.138:0/2326472916 learned_addr learned my addr 172.21.15.138:0/2326472916 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.885+0000 7f14c884c640 1 -- 172.21.15.138:0/2326472916 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd38d2680 con 0x55fcd4645c00 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.885+0000 7f14c884c640 1 --2- 172.21.15.138:0/2326472916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55fcd47d6780 tx=0x55fcd3bac0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9605964db989c1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.885+0000 7f14af81a640 1 -- 172.21.15.138:0/2326472916 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd4646580 con 0x55fcd4645c00 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.885+0000 7f14af81a640 1 -- 172.21.15.138:0/2326472916 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd4732000 con 0x55fcd4645c00 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.885+0000 7f14af81a640 1 -- 172.21.15.138:0/2326472916 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd46466e0 con 0x55fcd4645c00 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2326472916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 msgr2=0x55fcd3b44680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2326472916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55fcd47d6780 tx=0x55fcd3bac0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2326472916 shutdown_connections 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2326472916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2326472916 >> 172.21.15.138:0/2326472916 conn(0x55fcd4645400 msgr2=0x55fcd396b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2326472916 shutdown_connections 2024-04-27T05:21:34.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2326472916 wait complete. 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 Processor -- start 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- start start 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14ccc0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcd3884500 con 0x55fcd4645c00 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14c884c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14c884c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42968/0 (socket says 172.21.15.138:42968) 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.886+0000 7f14c884c640 1 -- 172.21.15.138:0/2062743803 learned_addr learned my addr 172.21.15.138:0/2062743803 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14c884c640 1 -- 172.21.15.138:0/2062743803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcd38d24e0 con 0x55fcd4645c00 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14c884c640 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55fcd3bc7740 tx=0x55fcd3bac000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14ae017640 1 -- 172.21.15.138:0/2062743803 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd4646840 con 0x55fcd4645c00 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14ae017640 1 -- 172.21.15.138:0/2062743803 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fcd4732780 con 0x55fcd4645c00 2024-04-27T05:21:34.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fcd3864ea0 con 0x55fcd4645c00 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14ae017640 1 -- 172.21.15.138:0/2062743803 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fcd4646580 con 0x55fcd4645c00 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcd38656c0 con 0x55fcd4645c00 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.887+0000 7f14ae017640 1 -- 172.21.15.138:0/2062743803 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fcd388ab00 con 0x55fcd4645c00 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ae017640 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd4644800 0x55fcd3b45700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ae017640 1 -- 172.21.15.138:0/2062743803 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55fcd3873dc0 con 0x55fcd4645c00 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ccc0f880 5 note: GC not initialized 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14c984e640 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd4644800 0x55fcd3b45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ab812640 20 reqs_thread_entry: start 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ccc0f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ccc0f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14c984e640 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd4644800 0x55fcd3b45700 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55fcd47d6ba0 tx=0x55fcd3bac960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14a980e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:34.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14a8fec640 20 reqs_thread_entry: start 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ccc0f880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.888+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4000 con 0x55fcd4659c00 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4400 con 0x55fcd4659c00 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.889+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4c00 con 0x55fcd4659c00 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~513] v29'6 uv6 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14ccc0f880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14ccc0f880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14ccc0f880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94544371897856 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd5000 con 0x55fcd4659c00 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== watch-notify(notify (1) cookie 94544371890944 notify 124554051589 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x55fcd3babd40 con 0x55fcd4659c00 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c3842640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051589 cookie 94544371890944 notifier 4381 bl.length()=689 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c3842640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:34.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c3842640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify cookie 94544371897856 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.890+0000 7f14c3842640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644c00 con 0x55fcd4659c00 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.891+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.891+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== watch-notify(notify_complete (2) cookie 94544371897856 notify 124554051589 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55fcd3c5d860 con 0x55fcd4659c00 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.891+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd5400 con 0x55fcd4659c00 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14ccc0f880 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14ccc0f880 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14ccc0f880 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94544371897856 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd5800 con 0x55fcd3be4800 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== watch-notify(notify (1) cookie 94544371886336 notify 124554051586 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x55fcd3c5dba0 con 0x55fcd3be4800 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c3842640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051586 cookie 94544371886336 notifier 4381 bl.length()=166 2024-04-27T05:21:34.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c3842640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c3842640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.0 [notify cookie 94544371897856 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c3842640 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd4644400 con 0x55fcd3be4800 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.892+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== watch-notify(notify_complete (2) cookie 94544371897856 notify 124554051586 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55fcd3c5dd40 con 0x55fcd3be4800 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo1", 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "tCJhOsEgCrjY7t4Ev5CabUkFBWcD9DtpMw5GFIzi" 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:34.923 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.893+0000 7f14b001b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.893+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94544371895552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd5c00 con 0x55fcd4659c00 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.894+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94544371895552] v29'77 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.894+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94544371886336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4000 con 0x55fcd3be4800 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.895+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94544371886336] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.895+0000 7f14ccc0f880 20 remove_watcher() i=0 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.895+0000 7f14ccc0f880 2 removed watcher, disabling cache 2024-04-27T05:21:34.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.895+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94544371887488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4400 con 0x55fcd4659c00 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.896+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94544371887488] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.896+0000 7f14ccc0f880 20 remove_watcher() i=1 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.896+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94544371888640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4c00 con 0x55fcd3be5400 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.897+0000 7f14c904d640 1 -- 172.21.15.138:0/4149148856 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94544371888640] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6d80 con 0x55fcd3be5400 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.897+0000 7f14ccc0f880 20 remove_watcher() i=2 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.897+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94544371889792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd4800 con 0x55fcd3be4800 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.898+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94544371889792] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf6fc0 con 0x55fcd3be4800 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.898+0000 7f14ccc0f880 20 remove_watcher() i=3 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.898+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94544371890944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3cd5000 con 0x55fcd4659c00 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.899+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94544371890944] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd4659c00 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.899+0000 7f14ccc0f880 20 remove_watcher() i=4 2024-04-27T05:21:34.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.899+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94544371892096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3b9e000 con 0x55fcd3be4400 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.900+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94544371892096] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.900+0000 7f14ccc0f880 20 remove_watcher() i=5 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.900+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94544371893248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3b9e400 con 0x55fcd3be4400 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.901+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94544371893248] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.901+0000 7f14ccc0f880 20 remove_watcher() i=6 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.901+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94544371894400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fcd3b9e800 con 0x55fcd3be4400 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14c984e640 1 -- 172.21.15.138:0/4149148856 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94544371894400] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fcd3bf66c0 con 0x55fcd3be4400 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 20 remove_watcher() i=7 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fcd3be4400 msgr2=0x55fcd3b45180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fcd3be4400 0x55fcd3b45180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fcd4659c00 msgr2=0x55fcd3b43b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fcd4659c00 0x55fcd3b43b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fcd3be4800 msgr2=0x55fcd3b44c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fcd3be4800 0x55fcd3b44c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fcd3be5400 msgr2=0x55fcd3b44100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fcd3be5400 0x55fcd3b44100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd3be4000 msgr2=0x55fcd3b43600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd3be4000 0x55fcd3b43600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55fcd396cc00 tx=0x55fcd3bac120 comp rx=0 tx=0).stop 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 msgr2=0x55fcd3b42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/4149148856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4659800 0x55fcd3b42b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55fcd4665470 tx=0x55fcd39731a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 reap_dead start 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14c884c640 1 -- 172.21.15.138:0/4149148856 reap_dead start 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 shutdown_connections 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 >> 172.21.15.138:0/4149148856 conn(0x55fcd3861400 msgr2=0x55fcd396b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 shutdown_connections 2024-04-27T05:21:34.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.902+0000 7f14ccc0f880 1 -- 172.21.15.138:0/4149148856 wait complete. 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd4644800 msgr2=0x55fcd3b45700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd4644800 0x55fcd3b45700 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55fcd47d6ba0 tx=0x55fcd3bac960 comp rx=0 tx=0).stop 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 msgr2=0x55fcd3b44680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55fcd3bc7740 tx=0x55fcd3bac000 comp rx=0 tx=0).stop 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 shutdown_connections 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd4644800 0x55fcd3b45700 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2062743803 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd4645c00 0x55fcd3b44680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 >> 172.21.15.138:0/2062743803 conn(0x55fcd4645400 msgr2=0x55fcd396b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 shutdown_connections 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2062743803 wait complete. 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd47b6000 msgr2=0x55fcd3b43080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd47b6000 0x55fcd3b43080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55fcd4766450 tx=0x55fcd39734a0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 msgr2=0x55fcd3b42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55fcd471cea0 tx=0x55fcd3972c60 comp rx=0 tx=0).stop 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 shutdown_connections 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fcd47b6000 0x55fcd3b43080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 --2- 172.21.15.138:0/2116225708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcd3861000 0x55fcd3b42580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 >> 172.21.15.138:0/2116225708 conn(0x55fcd3860c00 msgr2=0x55fcd3969000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 shutdown_connections 2024-04-27T05:21:34.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.903+0000 7f14ccc0f880 1 -- 172.21.15.138:0/2116225708 wait complete. 2024-04-27T05:21:34.934 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'tCJhOsEgCrjY7t4Ev5CabUkFBWcD9DtpMw5GFIzi'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:34.934 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-27T05:21:34.935 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-27T05:21:34.935 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-04-27T05:21:34.981 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:34.981 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:34.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.979+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.979+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.979+0000 7f99acb33880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.979+0000 7f99acb33880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5580ff42c600 con 0x5580ff155400 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.979+0000 7f99acb33880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5580ff194280 con 0x5580ff155000 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.980+0000 7f99a9422640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.980+0000 7f99a8c21640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5580ff155400 0x5580ff25c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60816/0 (socket says 172.21.15.138:60816) 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.980+0000 7f99a9422640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42982/0 (socket says 172.21.15.138:42982) 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.980+0000 7f99a9422640 1 -- 172.21.15.138:0/1922878819 learned_addr learned my addr 172.21.15.138:0/1922878819 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.980+0000 7f99a9422640 1 -- 172.21.15.138:0/1922878819 >> v1:172.21.15.114:6789/0 conn(0x5580ff155400 legacy=0x5580ff25c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.980+0000 7f99a9422640 1 -- 172.21.15.138:0/1922878819 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff159380 con 0x5580ff155000 2024-04-27T05:21:34.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.981+0000 7f99a9422640 1 --2- 172.21.15.138:0/1922878819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5580ff260d50 tx=0x5580ff266de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4160fee5887a6880 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99a8420640 1 -- 172.21.15.138:0/1922878819 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16dce0 con 0x5580ff155000 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99a8420640 1 -- 172.21.15.138:0/1922878819 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580ff42db00 con 0x5580ff155000 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99a8420640 1 -- 172.21.15.138:0/1922878819 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16de40 con 0x5580ff155000 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 -- 172.21.15.138:0/1922878819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 msgr2=0x5580ff436580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 --2- 172.21.15.138:0/1922878819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5580ff260d50 tx=0x5580ff266de0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 -- 172.21.15.138:0/1922878819 shutdown_connections 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 --2- 172.21.15.138:0/1922878819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 -- 172.21.15.138:0/1922878819 >> 172.21.15.138:0/1922878819 conn(0x5580ff154c00 msgr2=0x5580ff25c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 -- 172.21.15.138:0/1922878819 shutdown_connections 2024-04-27T05:21:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.982+0000 7f99acb33880 1 -- 172.21.15.138:0/1922878819 wait complete. 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.983+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.983+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.983+0000 7f99acb33880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99acb33880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ff194780 con 0x5580ff155000 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a9422640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a9422640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:42988/0 (socket says 172.21.15.138:42988) 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a9422640 1 -- 172.21.15.138:0/656451424 learned_addr learned my addr 172.21.15.138:0/656451424 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a9422640 1 -- 172.21.15.138:0/656451424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff158ea0 con 0x5580ff155000 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a9422640 1 --2- 172.21.15.138:0/656451424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5580fff59560 tx=0x5580ff266cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaab511ae035cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a7c1f640 1 -- 172.21.15.138:0/656451424 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16d4a0 con 0x5580ff155000 2024-04-27T05:21:35.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a7c1f640 1 -- 172.21.15.138:0/656451424 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5580ff42de00 con 0x5580ff155000 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.984+0000 7f99a7c1f640 1 -- 172.21.15.138:0/656451424 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16dce0 con 0x5580ff155000 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- 172.21.15.138:0/656451424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 msgr2=0x5580ff436580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 --2- 172.21.15.138:0/656451424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x5580fff59560 tx=0x5580ff266cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- 172.21.15.138:0/656451424 shutdown_connections 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 --2- 172.21.15.138:0/656451424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- 172.21.15.138:0/656451424 >> 172.21.15.138:0/656451424 conn(0x5580ff154c00 msgr2=0x5580ff25d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- 172.21.15.138:0/656451424 shutdown_connections 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- 172.21.15.138:0/656451424 wait complete. 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:35.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99acb33880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ff1948c0 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99a9422640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99a9422640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43002/0 (socket says 172.21.15.138:43002) 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.985+0000 7f99a9422640 1 -- 172.21.15.138:0/4206666546 learned_addr learned my addr 172.21.15.138:0/4206666546 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99a9422640 1 -- 172.21.15.138:0/4206666546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff1596c0 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99a9422640 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5581000112c0 tx=0x5580ff266c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99a641c640 1 -- 172.21.15.138:0/4206666546 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16de40 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99a641c640 1 -- 172.21.15.138:0/4206666546 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558100012000 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5580ff1591e0 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99a641c640 1 -- 172.21.15.138:0/4206666546 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16d4a0 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580ff1c6ea0 con 0x5580ff155000 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:35.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.986+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99acb33880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99acb33880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ff178280 con 0x5580fff4cc00 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a641c640 1 -- 172.21.15.138:0/4206666546 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5580ff17edc0 con 0x5580ff155000 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a641c640 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580fff4dc00 0x5580ff437080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9c23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9c23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43016/0 (socket says 172.21.15.138:43016) 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9c23640 1 -- 172.21.15.138:0/3367598626 learned_addr learned my addr 172.21.15.138:0/3367598626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a641c640 1 -- 172.21.15.138:0/4206666546 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5580ff167a40 con 0x5580ff155000 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9422640 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580fff4dc00 0x5580ff437080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9c23640 1 -- 172.21.15.138:0/3367598626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff1c6d00 con 0x5580fff4cc00 2024-04-27T05:21:35.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9c23640 1 --2- 172.21.15.138:0/3367598626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5580fff595f0 tx=0x5580ff267c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d075e65c9ad355 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a9422640 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580fff4dc00 0x5580ff437080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x558100044780 tx=0x5580ff266ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a3c17640 1 -- 172.21.15.138:0/3367598626 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16de40 con 0x5580fff4cc00 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a3c17640 1 -- 172.21.15.138:0/3367598626 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558100012600 con 0x5580fff4cc00 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.987+0000 7f99a3c17640 1 -- 172.21.15.138:0/3367598626 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16d4a0 con 0x5580fff4cc00 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- 172.21.15.138:0/3367598626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 msgr2=0x5580ff436b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 --2- 172.21.15.138:0/3367598626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5580fff595f0 tx=0x5580ff267c20 comp rx=0 tx=0).stop 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- 172.21.15.138:0/3367598626 shutdown_connections 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 --2- 172.21.15.138:0/3367598626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- 172.21.15.138:0/3367598626 >> 172.21.15.138:0/3367598626 conn(0x5580ff155400 msgr2=0x5580ff25f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- 172.21.15.138:0/3367598626 shutdown_connections 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- 172.21.15.138:0/3367598626 wait complete. 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:35.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99acb33880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ff179180 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99a9c23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99a9c23640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43032/0 (socket says 172.21.15.138:43032) 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.988+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 learned_addr learned my addr 172.21.15.138:0/1144703156 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff1c6b60 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99a9c23640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x558100011350 tx=0x5580ff2679e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99a2414640 1 -- 172.21.15.138:0/1144703156 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16de40 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99a2414640 1 -- 172.21.15.138:0/1144703156 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558100012780 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5580ff1c69c0 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99a2414640 1 -- 172.21.15.138:0/1144703156 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff16d4a0 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.989+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580ff1c6820 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a2414640 1 -- 172.21.15.138:0/1144703156 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5580ff17eb00 con 0x5580fff4cc00 2024-04-27T05:21:35.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a2414640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580ff4d2000 0x5580ff437600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a2414640 1 -- 172.21.15.138:0/1144703156 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5580ff167c00 con 0x5580fff4cc00 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a8c21640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580ff4d2000 0x5580ff437600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5580fff4d000 0x5580ff437b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580fff4d400 con 0x5580fff4d000 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a9c23640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5580fff4d000 0x5580ff437b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a8c21640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580ff4d2000 0x5580ff437600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5580ff261380 tx=0x5580ff49a120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.990+0000 7f99a9c23640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5580fff4d000 0x5580ff437b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 realm 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580fff4d800 con 0x5580fff4d000 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3000 con 0x5580fff4d000 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3400 con 0x5580fff4d000 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5580ff4d3800 0x5580ff438100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3c00 con 0x5580ff4d3800 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.991+0000 7f99a9422640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5580ff4d3800 0x5580ff438100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99a9422640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5580ff4d3800 0x5580ff438100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99acb33880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99acb33880 20 searching for the correct realm 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3000 con 0x5580ff4d3800 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99a9422640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5580ff4d3000 0x5580ff438c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3c00 con 0x5580ff4d3000 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.992+0000 7f99a8c21640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5580ff4d3000 0x5580ff438c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.993+0000 7f99a8c21640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5580ff4d3000 0x5580ff438c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.993+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.993+0000 7f99a8c21640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5580ff4d3c00 0x5580ff439180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.993+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.993+0000 7f99a9c23640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5580ff4d3c00 0x5580ff439180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.993+0000 7f99a9c23640 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5580ff4d3c00 0x5580ff439180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.994+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3000 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.995+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3000 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.996+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.997+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.998+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3000 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3800 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:34.999+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3c00 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580fff4d000 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d2c00 con 0x5580ff4d3000 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3400 con 0x5580fff4d000 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580fff4d400 con 0x5580fff4d000 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.000+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580fff4d800 con 0x5580ff4d3800 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 20 zone default found 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 4 Realm: () 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 20 zonegroup default 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc000 con 0x5580ff4d3c00 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc400 con 0x5580ff4d3800 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.001+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc800 con 0x5580ff4d3800 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 20 started sync module instance, tier type = 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.002+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fcc00 con 0x5580ff4d3000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.003+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.003+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94012526749952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fd000 con 0x5580ff4d3000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.003+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fd400 con 0x5580fff4d000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.004+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94012526749952] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.004+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.004+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94012526751104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fd800 con 0x5580fff4d000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.004+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fdc00 con 0x5580ff4d3800 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.005+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94012526751104] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.005+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.005+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94012526752256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc400 con 0x5580ff4d3800 2024-04-27T05:21:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.005+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc800 con 0x5580ff4d3000 2024-04-27T05:21:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.006+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94012526752256] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.006+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.006+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94012526753408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fdc00 con 0x5580ff4d3000 2024-04-27T05:21:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.006+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc000 con 0x5580fff4d000 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.007+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.007+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94012526754560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580fff4d800 con 0x5580fff4d000 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.007+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580fff4d400 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.007+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94012526753408] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.008+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.008+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94012526755712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4d3400 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.008+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581000b5800 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.008+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94012526754560] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.009+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94012526755712] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.009+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.009+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94012526756864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5581000b5400 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.009+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fcc00 con 0x5580ff4d3c00 2024-04-27T05:21:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94012526756864] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94012526758016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fd000 con 0x5580ff4d3c00 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=0 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=1 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=2 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=3 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=4 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=5 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.010+0000 7f99acb33880 20 add_watcher() i=6 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.011+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94012526758016] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.011+0000 7f99acb33880 20 add_watcher() i=7 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.011+0000 7f99acb33880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.011+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fc800 con 0x5580fff4d000 2024-04-27T05:21:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.011+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.011+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94012526759168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff4fd400 con 0x5580fff4d000 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.012+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94012526759168] v29'78 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.012+0000 7f99903f0640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.012+0000 7f99acb33880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.012+0000 7f99acb33880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99acb33880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99acb33880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ff178780 con 0x5580ff4d2c00 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99a8c21640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99a8c21640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43034/0 (socket says 172.21.15.138:43034) 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99a8c21640 1 -- 172.21.15.138:0/2176639843 learned_addr learned my addr 172.21.15.138:0/2176639843 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.013+0000 7f99a8c21640 1 -- 172.21.15.138:0/2176639843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff1c6680 con 0x5580ff4d2c00 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f99a8c21640 1 --2- 172.21.15.138:0/2176639843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5580ff4b4b40 tx=0x5580ff49a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686a11157568a410 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f998fbef640 1 -- 172.21.15.138:0/2176639843 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff4fe580 con 0x5580ff4d2c00 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f998fbef640 1 -- 172.21.15.138:0/2176639843 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558100012000 con 0x5580ff4d2c00 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f998fbef640 1 -- 172.21.15.138:0/2176639843 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff4fe6e0 con 0x5580ff4d2c00 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f99acb33880 1 -- 172.21.15.138:0/2176639843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 msgr2=0x5580ff438680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f99acb33880 1 --2- 172.21.15.138:0/2176639843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5580ff4b4b40 tx=0x5580ff49a180 comp rx=0 tx=0).stop 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f99acb33880 1 -- 172.21.15.138:0/2176639843 shutdown_connections 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f99acb33880 1 --2- 172.21.15.138:0/2176639843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.014+0000 7f99acb33880 1 -- 172.21.15.138:0/2176639843 >> 172.21.15.138:0/2176639843 conn(0x5580ff4fd800 msgr2=0x5580ff25f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99acb33880 1 -- 172.21.15.138:0/2176639843 shutdown_connections 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99acb33880 1 -- 172.21.15.138:0/2176639843 wait complete. 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99acb33880 1 Processor -- start 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99acb33880 1 -- start start 2024-04-27T05:21:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99acb33880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99acb33880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580ff178500 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99a8c21640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99a8c21640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43048/0 (socket says 172.21.15.138:43048) 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99a8c21640 1 -- 172.21.15.138:0/3303864319 learned_addr learned my addr 172.21.15.138:0/3303864319 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99a8c21640 1 -- 172.21.15.138:0/3303864319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580ff1c64e0 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.015+0000 7f99a8c21640 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5580ff4b5860 tx=0x5580ff49aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f998e3ec640 1 -- 172.21.15.138:0/3303864319 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff4fe840 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f998e3ec640 1 -- 172.21.15.138:0/3303864319 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558100012a80 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5580ff159380 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f998e3ec640 1 -- 172.21.15.138:0/3303864319 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5580ff4fe580 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580ff158ea0 con 0x5580ff4d2c00 2024-04-27T05:21:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f998e3ec640 1 -- 172.21.15.138:0/3303864319 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5580ff17f8c0 con 0x5580ff4d2c00 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f998e3ec640 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581000b5800 0x5580ff439700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.016+0000 7f998e3ec640 1 -- 172.21.15.138:0/3303864319 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5580ff167dc0 con 0x5580ff4d2c00 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99acb33880 5 note: GC not initialized 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99a9c23640 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581000b5800 0x5580ff439700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f998bbe7640 20 reqs_thread_entry: start 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99acb33880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99a9c23640 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581000b5800 0x5580ff439700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5581000c2ae0 tx=0x5580ff49a8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99acb33880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f9989be3640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99893c1640 20 reqs_thread_entry: start 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99acb33880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99acb33880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.017+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5ce000 con 0x5580fff4d000 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.018+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv6 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.018+0000 7f99acb33880 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.018+0000 7f99acb33880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.018+0000 7f99acb33880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.018+0000 7f99acb33880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.018+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5ce800 con 0x5580fff4d000 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v29'7 uv7 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99acb33880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99acb33880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99acb33880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94012526761472 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5cec00 con 0x5580fff4d000 2024-04-27T05:21:35.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94012526754560 notify 124554051590 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x5580ff499d40 con 0x5580fff4d000 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a3c17640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051590 cookie 94012526754560 notifier 4389 bl.length()=689 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a3c17640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a3c17640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 94012526761472 out=8b] v0'0 uv57 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a3c17640 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5de000 con 0x5580fff4d000 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.019+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94012526761472 notify 124554051590 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5580ff555a00 con 0x5580fff4d000 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo1", 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.047 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.020+0000 7f99903f0640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.020+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94012526759168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5cf000 con 0x5580fff4d000 2024-04-27T05:21:35.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.021+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94012526759168] v29'79 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.021+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94012526749952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5cf400 con 0x5580ff4d3000 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.022+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94012526749952] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.022+0000 7f99acb33880 20 remove_watcher() i=0 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.022+0000 7f99acb33880 2 removed watcher, disabling cache 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.022+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94012526751104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5cf800 con 0x5580fff4d000 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.023+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94012526751104] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.023+0000 7f99acb33880 20 remove_watcher() i=1 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.023+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94012526752256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5cfc00 con 0x5580ff4d3800 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.024+0000 7f99a9422640 1 -- 172.21.15.138:0/1144703156 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94012526752256] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2d80 con 0x5580ff4d3800 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.024+0000 7f99acb33880 20 remove_watcher() i=2 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.024+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94012526753408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5de000 con 0x5580ff4d3000 2024-04-27T05:21:35.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.025+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94012526753408] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f2fc0 con 0x5580ff4d3000 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.025+0000 7f99acb33880 20 remove_watcher() i=3 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.025+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94012526754560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5de800 con 0x5580fff4d000 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.026+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94012526754560] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580fff4d000 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.026+0000 7f99acb33880 20 remove_watcher() i=4 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.026+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94012526755712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5dec00 con 0x5580ff4d3c00 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.027+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94012526755712] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.027+0000 7f99acb33880 20 remove_watcher() i=5 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.027+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94012526756864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5df000 con 0x5580ff4d3c00 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.028+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94012526756864] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.028+0000 7f99acb33880 20 remove_watcher() i=6 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.028+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94012526758016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5580ff5df400 con 0x5580ff4d3c00 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99a9c23640 1 -- 172.21.15.138:0/1144703156 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94012526758016] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5580ff4f26c0 con 0x5580ff4d3c00 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 20 remove_watcher() i=7 2024-04-27T05:21:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5580ff4d3c00 msgr2=0x5580ff439180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5580ff4d3c00 0x5580ff439180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5580fff4d000 msgr2=0x5580ff437b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5580fff4d000 0x5580ff437b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5580ff4d3000 msgr2=0x5580ff438c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5580ff4d3000 0x5580ff438c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5580ff4d3800 msgr2=0x5580ff438100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5580ff4d3800 0x5580ff438100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580ff4d2000 msgr2=0x5580ff437600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580ff4d2000 0x5580ff437600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5580ff261380 tx=0x5580ff49a120 comp rx=0 tx=0).stop 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 msgr2=0x5580ff436b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/1144703156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580fff4cc00 0x5580ff436b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x558100011350 tx=0x5580ff2679e0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 reap_dead start 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99a8c21640 1 -- 172.21.15.138:0/1144703156 reap_dead start 2024-04-27T05:21:35.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 shutdown_connections 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 >> 172.21.15.138:0/1144703156 conn(0x5580ff155400 msgr2=0x5580ff25f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 shutdown_connections 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/1144703156 wait complete. 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581000b5800 msgr2=0x5580ff439700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581000b5800 0x5580ff439700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5581000c2ae0 tx=0x5580ff49a8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 msgr2=0x5580ff438680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5580ff4b5860 tx=0x5580ff49aae0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 shutdown_connections 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5581000b5800 0x5580ff439700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 --2- 172.21.15.138:0/3303864319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff4d2c00 0x5580ff438680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.029+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 >> 172.21.15.138:0/3303864319 conn(0x5580ff4fd800 msgr2=0x5580ff25f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 shutdown_connections 2024-04-27T05:21:35.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/3303864319 wait complete. 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580fff4dc00 msgr2=0x5580ff437080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580fff4dc00 0x5580ff437080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x558100044780 tx=0x5580ff266ea0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 msgr2=0x5580ff436580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5581000112c0 tx=0x5580ff266c60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 shutdown_connections 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5580fff4dc00 0x5580ff437080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 --2- 172.21.15.138:0/4206666546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5580ff155000 0x5580ff436580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 >> 172.21.15.138:0/4206666546 conn(0x5580ff154c00 msgr2=0x5580ff25d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 shutdown_connections 2024-04-27T05:21:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.030+0000 7f99acb33880 1 -- 172.21.15.138:0/4206666546 wait complete. 2024-04-27T05:21:35.057 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.058 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:35.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:35.058 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:35.100 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.100 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.098+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.098+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.098+0000 7fb0f0b36880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.098+0000 7fb0f0b36880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a37b214600 con 0x55a37af3d400 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.098+0000 7fb0f0b36880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a37af7c280 con 0x55a37af3d000 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.099+0000 7fb0ed4a2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.099+0000 7fb0ed4a2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43058/0 (socket says 172.21.15.138:43058) 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.099+0000 7fb0ecca1640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a37af3d400 0x55a37b044800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60822/0 (socket says 172.21.15.138:60822) 2024-04-27T05:21:35.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.099+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/833107653 learned_addr learned my addr 172.21.15.138:0/833107653 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.099+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/833107653 >> v1:172.21.15.114:6789/0 conn(0x55a37af3d400 legacy=0x55a37b044800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.099+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/833107653 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37af41380 con 0x55a37af3d000 2024-04-27T05:21:35.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/833107653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55a37b048d50 tx=0x55a37b04ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b25718801edd3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0ec4a0640 1 -- 172.21.15.138:0/833107653 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af55ce0 con 0x55a37af3d000 2024-04-27T05:21:35.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0ec4a0640 1 -- 172.21.15.138:0/833107653 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37b215b00 con 0x55a37af3d000 2024-04-27T05:21:35.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0ec4a0640 1 -- 172.21.15.138:0/833107653 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af55e40 con 0x55a37af3d000 2024-04-27T05:21:35.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 -- 172.21.15.138:0/833107653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 msgr2=0x55a37b21e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/833107653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55a37b048d50 tx=0x55a37b04ede0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 -- 172.21.15.138:0/833107653 shutdown_connections 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/833107653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 -- 172.21.15.138:0/833107653 >> 172.21.15.138:0/833107653 conn(0x55a37af3cc00 msgr2=0x55a37b044000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 -- 172.21.15.138:0/833107653 shutdown_connections 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.101+0000 7fb0f0b36880 1 -- 172.21.15.138:0/833107653 wait complete. 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0f0b36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0f0b36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37af7c780 con 0x55a37af3d000 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0ed4a2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0ed4a2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43066/0 (socket says 172.21.15.138:43066) 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.102+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2125447103 learned_addr learned my addr 172.21.15.138:0/2125447103 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2125447103 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37af40ea0 con 0x55a37af3d000 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/2125447103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a37bd41560 tx=0x55a37b04ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52effbb4c765a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0ebc9f640 1 -- 172.21.15.138:0/2125447103 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af554a0 con 0x55a37af3d000 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0ebc9f640 1 -- 172.21.15.138:0/2125447103 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37b215e00 con 0x55a37af3d000 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0ebc9f640 1 -- 172.21.15.138:0/2125447103 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af55ce0 con 0x55a37af3d000 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2125447103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 msgr2=0x55a37b21e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2125447103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a37bd41560 tx=0x55a37b04ecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2125447103 shutdown_connections 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2125447103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2125447103 >> 172.21.15.138:0/2125447103 conn(0x55a37af3cc00 msgr2=0x55a37b045000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2125447103 shutdown_connections 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.103+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2125447103 wait complete. 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0f0b36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0f0b36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37af7c8c0 con 0x55a37af3d000 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0ed4a2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0ed4a2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43072/0 (socket says 172.21.15.138:43072) 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/1630276028 learned_addr learned my addr 172.21.15.138:0/1630276028 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/1630276028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37af416c0 con 0x55a37af3d000 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55a37bdf92c0 tx=0x55a37b04ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.104+0000 7fb0ea49c640 1 -- 172.21.15.138:0/1630276028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af55e40 con 0x55a37af3d000 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0ea49c640 1 -- 172.21.15.138:0/1630276028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37bdfa000 con 0x55a37af3d000 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a37af411e0 con 0x55a37af3d000 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0ea49c640 1 -- 172.21.15.138:0/1630276028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af554a0 con 0x55a37af3d000 2024-04-27T05:21:35.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a37afaeea0 con 0x55a37af3d000 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0ea49c640 1 -- 172.21.15.138:0/1630276028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a37af66dc0 con 0x55a37af3d000 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0f0b36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0f0b36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37af7cc80 con 0x55a37bd34c00 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0ea49c640 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37bd35c00 0x55a37b21f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0ea49c640 1 -- 172.21.15.138:0/1630276028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a37af4fa40 con 0x55a37af3d000 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.105+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37bd35c00 0x55a37b21f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43088/0 (socket says 172.21.15.138:43088) 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0edca3640 1 -- 172.21.15.138:0/1820477279 learned_addr learned my addr 172.21.15.138:0/1820477279 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37bd35c00 0x55a37b21f080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55a37be3e180 tx=0x55a37b04ee40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0edca3640 1 -- 172.21.15.138:0/1820477279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37afaed00 con 0x55a37bd34c00 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0edca3640 1 --2- 172.21.15.138:0/1820477279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55a37bdf95f0 tx=0x55a37b04fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f168b52edcff3659 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0e7c97640 1 -- 172.21.15.138:0/1820477279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af55e40 con 0x55a37bd34c00 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0e7c97640 1 -- 172.21.15.138:0/1820477279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37bdfa600 con 0x55a37bd34c00 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0e7c97640 1 -- 172.21.15.138:0/1820477279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af554a0 con 0x55a37bd34c00 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1820477279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 msgr2=0x55a37b21eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1820477279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55a37bdf95f0 tx=0x55a37b04fc20 comp rx=0 tx=0).stop 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1820477279 shutdown_connections 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1820477279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.106+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1820477279 >> 172.21.15.138:0/1820477279 conn(0x55a37af3d400 msgr2=0x55a37b047000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1820477279 shutdown_connections 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1820477279 wait complete. 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0f0b36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0f0b36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37af60280 con 0x55a37bd34c00 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43100/0 (socket says 172.21.15.138:43100) 2024-04-27T05:21:35.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 learned_addr learned my addr 172.21.15.138:0/2988178788 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.107+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37afaeb60 con 0x55a37bd34c00 2024-04-27T05:21:35.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0edca3640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55a37bdf93b0 tx=0x55a37b04f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0e6494640 1 -- 172.21.15.138:0/2988178788 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af55e40 con 0x55a37bd34c00 2024-04-27T05:21:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0e6494640 1 -- 172.21.15.138:0/2988178788 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37bdfa780 con 0x55a37bd34c00 2024-04-27T05:21:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a37afae9c0 con 0x55a37bd34c00 2024-04-27T05:21:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0e6494640 1 -- 172.21.15.138:0/2988178788 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37af554a0 con 0x55a37bd34c00 2024-04-27T05:21:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a37afae820 con 0x55a37bd34c00 2024-04-27T05:21:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.108+0000 7fb0e6494640 1 -- 172.21.15.138:0/2988178788 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a37af66b00 con 0x55a37bd34c00 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0e6494640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37b2b6000 0x55a37b21f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0e6494640 1 -- 172.21.15.138:0/2988178788 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a37af4fc00 con 0x55a37bd34c00 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37b2b6000 0x55a37b21f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a37bd35000 0x55a37b21fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37bd35400 con 0x55a37bd35000 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0edca3640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a37bd35000 0x55a37b21fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37b2b6000 0x55a37b21f600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55a37b049380 tx=0x55a37b27c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0edca3640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a37bd35000 0x55a37b21fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 20 realm 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.109+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37bd35800 con 0x55a37bd35000 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7000 con 0x55a37bd35000 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7400 con 0x55a37bd35000 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a37b2b7800 0x55a37b220100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7c00 con 0x55a37b2b7800 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a37b2b7800 0x55a37b220100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.110+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a37b2b7800 0x55a37b220100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0f0b36880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0f0b36880 20 searching for the correct realm 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7000 con 0x55a37b2b7800 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0ed4a2640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a37b2b7000 0x55a37b220c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7c00 con 0x55a37b2b7000 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.111+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a37b2b7000 0x55a37b220c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.112+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a37b2b7000 0x55a37b220c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.112+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.112+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a37b2b7c00 0x55a37b221180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.112+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.112+0000 7fb0edca3640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a37b2b7c00 0x55a37b221180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a37b2b7c00 0x55a37b221180 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.113+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7000 2024-04-27T05:21:35.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.114+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7000 2024-04-27T05:21:35.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.115+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.116+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.116+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.116+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.116+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.116+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.116+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7000 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.117+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.118+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7000 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7800 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37bd35000 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.119+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b6c00 con 0x55a37b2b7000 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7400 con 0x55a37bd35000 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37bd35400 con 0x55a37bd35000 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37bd35800 con 0x55a37b2b7800 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 zone default found 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 4 Realm: () 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 zonegroup default 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.120+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0000 con 0x55a37b2b7c00 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0400 con 0x55a37b2b7800 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0800 con 0x55a37b2b7800 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 started sync module instance, tier type = 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.121+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0c00 con 0x55a37b2b7000 2024-04-27T05:21:35.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.122+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.122+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94160633817344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1000 con 0x55a37b2b7000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.122+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1400 con 0x55a37bd35000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.123+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94160633817344] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.123+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.123+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94160633818496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1800 con 0x55a37bd35000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.123+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1c00 con 0x55a37b2b7800 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.124+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.124+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94160633818496] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.124+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94160633819648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0400 con 0x55a37b2b7800 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.124+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0800 con 0x55a37b2b7000 2024-04-27T05:21:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.125+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.125+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94160633820800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1c00 con 0x55a37b2b7000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.125+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0000 con 0x55a37bd35000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.125+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94160633819648] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.126+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.126+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94160633821952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37bd35800 con 0x55a37bd35000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.126+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37bd35400 con 0x55a37b2b7c00 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.126+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94160633820800] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.127+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94160633821952] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.127+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.127+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94160633823104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2b7400 con 0x55a37b2b7c00 2024-04-27T05:21:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.127+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37be9ac00 con 0x55a37b2b7c00 2024-04-27T05:21:35.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.128+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94160633823104] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.128+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.128+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94160633824256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37be9a800 con 0x55a37b2b7c00 2024-04-27T05:21:35.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.128+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0c00 con 0x55a37b2b7c00 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94160633824256] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94160633825408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1000 con 0x55a37b2b7c00 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=0 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=1 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=2 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=3 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=4 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=5 2024-04-27T05:21:35.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.129+0000 7fb0f0b36880 20 add_watcher() i=6 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.130+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94160633825408] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.130+0000 7fb0f0b36880 20 add_watcher() i=7 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.130+0000 7fb0f0b36880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.130+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e0800 con 0x55a37bd35000 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.130+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.130+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94160633826560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b2e1400 con 0x55a37bd35000 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94160633826560] v29'80 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0d4470640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0f0b36880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0f0b36880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0f0b36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.131+0000 7fb0f0b36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37af608c0 con 0x55a37b2b6c00 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0ecca1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0ecca1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43116/0 (socket says 172.21.15.138:43116) 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0ecca1640 1 -- 172.21.15.138:0/3326503831 learned_addr learned my addr 172.21.15.138:0/3326503831 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0ecca1640 1 -- 172.21.15.138:0/3326503831 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37afae680 con 0x55a37b2b6c00 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/3326503831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a37b298ae0 tx=0x55a37b27c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65b671e5c33e718 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0d3c6f640 1 -- 172.21.15.138:0/3326503831 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37b2e2580 con 0x55a37b2b6c00 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0d3c6f640 1 -- 172.21.15.138:0/3326503831 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37bdfa000 con 0x55a37b2b6c00 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.132+0000 7fb0d3c6f640 1 -- 172.21.15.138:0/3326503831 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37b2e26e0 con 0x55a37b2b6c00 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- 172.21.15.138:0/3326503831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 msgr2=0x55a37b220680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/3326503831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55a37b298ae0 tx=0x55a37b27c180 comp rx=0 tx=0).stop 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- 172.21.15.138:0/3326503831 shutdown_connections 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/3326503831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- 172.21.15.138:0/3326503831 >> 172.21.15.138:0/3326503831 conn(0x55a37b2e1800 msgr2=0x55a37b047800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- 172.21.15.138:0/3326503831 shutdown_connections 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- 172.21.15.138:0/3326503831 wait complete. 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 Processor -- start 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- start start 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.133+0000 7fb0f0b36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a37af60780 con 0x55a37b2b6c00 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0ecca1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0ecca1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43130/0 (socket says 172.21.15.138:43130) 2024-04-27T05:21:35.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0ecca1640 1 -- 172.21.15.138:0/1586886034 learned_addr learned my addr 172.21.15.138:0/1586886034 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0ecca1640 1 -- 172.21.15.138:0/1586886034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a37afae4e0 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0ecca1640 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55a37b299860 tx=0x55a37b27cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0d246c640 1 -- 172.21.15.138:0/1586886034 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37b2e2840 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0d246c640 1 -- 172.21.15.138:0/1586886034 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a37bdfaa80 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a37af41380 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0d246c640 1 -- 172.21.15.138:0/1586886034 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a37b2e2580 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.134+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a37af40ea0 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0d246c640 1 -- 172.21.15.138:0/1586886034 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a37af678c0 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0d246c640 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37be9ac00 0x55a37b221700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0d246c640 1 -- 172.21.15.138:0/1586886034 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a37af4fdc0 con 0x55a37b2b6c00 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0f0b36880 5 note: GC not initialized 2024-04-27T05:21:35.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0edca3640 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37be9ac00 0x55a37b221700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0cfc67640 20 reqs_thread_entry: start 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0f0b36880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0edca3640 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37be9ac00 0x55a37b221700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55a37bea4c30 tx=0x55a37b27c8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0f0b36880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.135+0000 7fb0cdc63640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.136+0000 7fb0cd441640 20 reqs_thread_entry: start 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.136+0000 7fb0f0b36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.136+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bc000 con 0x55a37bd35000 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.136+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.136+0000 7fb0f0b36880 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:35.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo1", 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.171 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.172 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.173 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.137+0000 7fb0d4470640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.137+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94160633826560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bc400 con 0x55a37bd35000 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.138+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94160633826560] v29'81 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.138+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94160633817344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bc800 con 0x55a37b2b7000 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.139+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94160633817344] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.139+0000 7fb0f0b36880 20 remove_watcher() i=0 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.139+0000 7fb0f0b36880 2 removed watcher, disabling cache 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.139+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94160633818496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bcc00 con 0x55a37bd35000 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.140+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94160633818496] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.140+0000 7fb0f0b36880 20 remove_watcher() i=1 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.140+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94160633819648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bd000 con 0x55a37b2b7800 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.141+0000 7fb0ed4a2640 1 -- 172.21.15.138:0/2988178788 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94160633819648] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cefc0 con 0x55a37b2b7800 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.141+0000 7fb0f0b36880 20 remove_watcher() i=2 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.141+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94160633820800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bd400 con 0x55a37b2b7000 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.142+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94160633820800] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2cf200 con 0x55a37b2b7000 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.142+0000 7fb0f0b36880 20 remove_watcher() i=3 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.142+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94160633821952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bd800 con 0x55a37bd35000 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.143+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94160633821952] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37bd35000 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.143+0000 7fb0f0b36880 20 remove_watcher() i=4 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.143+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94160633823104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bdc00 con 0x55a37b2b7c00 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.144+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94160633823104] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.144+0000 7fb0f0b36880 20 remove_watcher() i=5 2024-04-27T05:21:35.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.144+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94160633824256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bc000 con 0x55a37b2b7c00 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.144+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94160633824256] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.144+0000 7fb0f0b36880 20 remove_watcher() i=6 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.144+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94160633825408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a37b3bc400 con 0x55a37b2b7c00 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0edca3640 1 -- 172.21.15.138:0/2988178788 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94160633825408] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a37b2ce900 con 0x55a37b2b7c00 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 20 remove_watcher() i=7 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a37b2b7c00 msgr2=0x55a37b221180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a37b2b7c00 0x55a37b221180 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a37bd35000 msgr2=0x55a37b21fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a37bd35000 0x55a37b21fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a37b2b7000 msgr2=0x55a37b220c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a37b2b7000 0x55a37b220c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a37b2b7800 msgr2=0x55a37b220100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a37b2b7800 0x55a37b220100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37b2b6000 msgr2=0x55a37b21f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37b2b6000 0x55a37b21f600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55a37b049380 tx=0x55a37b27c120 comp rx=0 tx=0).stop 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 msgr2=0x55a37b21eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/2988178788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37bd34c00 0x55a37b21eb00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55a37bdf93b0 tx=0x55a37b04f980 comp rx=0 tx=0).stop 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 reap_dead start 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0ecca1640 1 -- 172.21.15.138:0/2988178788 reap_dead start 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 shutdown_connections 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 >> 172.21.15.138:0/2988178788 conn(0x55a37af3d400 msgr2=0x55a37b047000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 shutdown_connections 2024-04-27T05:21:35.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.145+0000 7fb0f0b36880 1 -- 172.21.15.138:0/2988178788 wait complete. 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37be9ac00 msgr2=0x55a37b221700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37be9ac00 0x55a37b221700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55a37bea4c30 tx=0x55a37b27c8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 msgr2=0x55a37b220680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55a37b299860 tx=0x55a37b27cae0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 shutdown_connections 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37be9ac00 0x55a37b221700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1586886034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37b2b6c00 0x55a37b220680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 >> 172.21.15.138:0/1586886034 conn(0x55a37b2e1800 msgr2=0x55a37b047800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 shutdown_connections 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1586886034 wait complete. 2024-04-27T05:21:35.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37bd35c00 msgr2=0x55a37b21f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37bd35c00 0x55a37b21f080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55a37be3e180 tx=0x55a37b04ee40 comp rx=0 tx=0).stop 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 msgr2=0x55a37b21e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55a37bdf92c0 tx=0x55a37b04ec60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 shutdown_connections 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a37bd35c00 0x55a37b21f080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 --2- 172.21.15.138:0/1630276028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a37af3d000 0x55a37b21e580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 >> 172.21.15.138:0/1630276028 conn(0x55a37af3cc00 msgr2=0x55a37b045000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 shutdown_connections 2024-04-27T05:21:35.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.146+0000 7fb0f0b36880 1 -- 172.21.15.138:0/1630276028 wait complete. 2024-04-27T05:21:35.185 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.186 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:35.186 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:35.186 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:35.262 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.263 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa72aa11880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562f2d331400 0x562f2d612580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa72aa11880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562f2d608600 con 0x562f2d331000 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa72aa11880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x562f2d370280 con 0x562f2d331400 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa7267da640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562f2d331400 0x562f2d612580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa7267da640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562f2d331400 0x562f2d612580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43134/0 (socket says 172.21.15.138:43134) 2024-04-27T05:21:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa7267da640 1 -- 172.21.15.138:0/3083250535 learned_addr learned my addr 172.21.15.138:0/3083250535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa726fdb640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x562f2d331000 0x562f2d438800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60824/0 (socket says 172.21.15.138:60824) 2024-04-27T05:21:35.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa7267da640 1 -- 172.21.15.138:0/3083250535 >> v1:172.21.15.114:6789/0 conn(0x562f2d331000 legacy=0x562f2d438800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.261+0000 7fa7267da640 1 -- 172.21.15.138:0/3083250535 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d335380 con 0x562f2d331400 2024-04-27T05:21:35.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.263+0000 7fa7267da640 1 --2- 172.21.15.138:0/3083250535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331400 0x562f2d612580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x562f2d43cc00 tx=0x562f2d442de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b1702547aef8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.263+0000 7fa725fd9640 1 -- 172.21.15.138:0/3083250535 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d349ce0 con 0x562f2d331400 2024-04-27T05:21:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa725fd9640 1 -- 172.21.15.138:0/3083250535 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2d609b00 con 0x562f2d331400 2024-04-27T05:21:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa725fd9640 1 -- 172.21.15.138:0/3083250535 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d349e40 con 0x562f2d331400 2024-04-27T05:21:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 -- 172.21.15.138:0/3083250535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331400 msgr2=0x562f2d612580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 --2- 172.21.15.138:0/3083250535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331400 0x562f2d612580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x562f2d43cc00 tx=0x562f2d442de0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 -- 172.21.15.138:0/3083250535 shutdown_connections 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 --2- 172.21.15.138:0/3083250535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331400 0x562f2d612580 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 -- 172.21.15.138:0/3083250535 >> 172.21.15.138:0/3083250535 conn(0x562f2d330c00 msgr2=0x562f2d438000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 -- 172.21.15.138:0/3083250535 shutdown_connections 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.264+0000 7fa72aa11880 1 -- 172.21.15.138:0/3083250535 wait complete. 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa72aa11880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa72aa11880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f2d370780 con 0x562f2d331000 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa726fdb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa726fdb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43140/0 (socket says 172.21.15.138:43140) 2024-04-27T05:21:35.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa726fdb640 1 -- 172.21.15.138:0/1374105229 learned_addr learned my addr 172.21.15.138:0/1374105229 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa726fdb640 1 -- 172.21.15.138:0/1374105229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d334ea0 con 0x562f2d331000 2024-04-27T05:21:35.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.265+0000 7fa726fdb640 1 --2- 172.21.15.138:0/1374105229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562f2e1ed290 tx=0x562f2d442cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa181c32a18aa3d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa7257d8640 1 -- 172.21.15.138:0/1374105229 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d3494a0 con 0x562f2d331000 2024-04-27T05:21:35.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa7257d8640 1 -- 172.21.15.138:0/1374105229 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2d609e00 con 0x562f2d331000 2024-04-27T05:21:35.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa7257d8640 1 -- 172.21.15.138:0/1374105229 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d349ce0 con 0x562f2d331000 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- 172.21.15.138:0/1374105229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 msgr2=0x562f2d612580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 --2- 172.21.15.138:0/1374105229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562f2e1ed290 tx=0x562f2d442cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- 172.21.15.138:0/1374105229 shutdown_connections 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 --2- 172.21.15.138:0/1374105229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- 172.21.15.138:0/1374105229 >> 172.21.15.138:0/1374105229 conn(0x562f2d330c00 msgr2=0x562f2d439000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- 172.21.15.138:0/1374105229 shutdown_connections 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- 172.21.15.138:0/1374105229 wait complete. 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.266+0000 7fa72aa11880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f2d3708c0 con 0x562f2d331000 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa726fdb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa726fdb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43154/0 (socket says 172.21.15.138:43154) 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa726fdb640 1 -- 172.21.15.138:0/3720239890 learned_addr learned my addr 172.21.15.138:0/3720239890 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa726fdb640 1 -- 172.21.15.138:0/3720239890 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d3356c0 con 0x562f2d331000 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa726fdb640 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x562f2e1ecb40 tx=0x562f2d442c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa723fd5640 1 -- 172.21.15.138:0/3720239890 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d349e40 con 0x562f2d331000 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa723fd5640 1 -- 172.21.15.138:0/3720239890 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2e202000 con 0x562f2d331000 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f2d3351e0 con 0x562f2d331000 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa723fd5640 1 -- 172.21.15.138:0/3720239890 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d3494a0 con 0x562f2d331000 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.267+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f2d3a2ea0 con 0x562f2d331000 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa72aa11880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa72aa11880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f2d370c80 con 0x562f2e129800 2024-04-27T05:21:35.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa723fd5640 1 -- 172.21.15.138:0/3720239890 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562f2d35b600 con 0x562f2d331000 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa723fd5640 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e286000 0x562f2d613080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa723fd5640 1 -- 172.21.15.138:0/3720239890 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x562f2d343a40 con 0x562f2d331000 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa7277dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa7277dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43162/0 (socket says 172.21.15.138:43162) 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa7277dc640 1 -- 172.21.15.138:0/1463343454 learned_addr learned my addr 172.21.15.138:0/1463343454 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.268+0000 7fa726fdb640 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e286000 0x562f2d613080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa7277dc640 1 -- 172.21.15.138:0/1463343454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d3a2d00 con 0x562f2e129800 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa726fdb640 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e286000 0x562f2d613080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x562f2e234510 tx=0x562f2d4434a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa7277dc640 1 --2- 172.21.15.138:0/1463343454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x562f2e1edad0 tx=0x562f2d443c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec082cc7e94b0e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa7217d0640 1 -- 172.21.15.138:0/1463343454 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d349e40 con 0x562f2e129800 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa7217d0640 1 -- 172.21.15.138:0/1463343454 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2e202300 con 0x562f2e129800 2024-04-27T05:21:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa7217d0640 1 -- 172.21.15.138:0/1463343454 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d3494a0 con 0x562f2e129800 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 -- 172.21.15.138:0/1463343454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 msgr2=0x562f2d612b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 --2- 172.21.15.138:0/1463343454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x562f2e1edad0 tx=0x562f2d443c20 comp rx=0 tx=0).stop 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 -- 172.21.15.138:0/1463343454 shutdown_connections 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 --2- 172.21.15.138:0/1463343454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 -- 172.21.15.138:0/1463343454 >> 172.21.15.138:0/1463343454 conn(0x562f2d331400 msgr2=0x562f2d43b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 -- 172.21.15.138:0/1463343454 shutdown_connections 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.269+0000 7fa72aa11880 1 -- 172.21.15.138:0/1463343454 wait complete. 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa72aa11880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa72aa11880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f2d354280 con 0x562f2e129800 2024-04-27T05:21:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa7277dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa7277dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43170/0 (socket says 172.21.15.138:43170) 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 learned_addr learned my addr 172.21.15.138:0/2525780728 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d3a2b60 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.270+0000 7fa7277dc640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x562f2e1ed920 tx=0x562f2d443a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa71ffcd640 1 -- 172.21.15.138:0/2525780728 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d349e40 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa71ffcd640 1 -- 172.21.15.138:0/2525780728 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2e202480 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f2d3a29c0 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa71ffcd640 1 -- 172.21.15.138:0/2525780728 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2d3494a0 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f2d3a2820 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa71ffcd640 1 -- 172.21.15.138:0/2525780728 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562f2d35bb80 con 0x562f2e129800 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.271+0000 7fa71ffcd640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2d6c4000 0x562f2d613600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa71ffcd640 1 -- 172.21.15.138:0/2525780728 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x562f2d343c00 con 0x562f2e129800 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa7267da640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2d6c4000 0x562f2d613600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562f2e129c00 0x562f2d613b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4400 con 0x562f2e129c00 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa7277dc640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562f2e129c00 0x562f2d613b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa7267da640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2d6c4000 0x562f2d613600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x562f2d43cc00 tx=0x562f2d688120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa7277dc640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562f2e129c00 0x562f2d613b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa72aa11880 20 realm 2024-04-27T05:21:35.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4800 con 0x562f2e129c00 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.272+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4c00 con 0x562f2e129c00 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5000 con 0x562f2e129c00 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562f2d6c5400 0x562f2d614100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4400 con 0x562f2d6c5400 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa726fdb640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562f2d6c5400 0x562f2d614100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.273+0000 7fa726fdb640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562f2d6c5400 0x562f2d614100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa72aa11880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa72aa11880 20 searching for the correct realm 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4800 con 0x562f2d6c5400 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa726fdb640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562f2d6c4800 0x562f2d614c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4400 con 0x562f2d6c4800 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa7267da640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562f2d6c4800 0x562f2d614c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.274+0000 7fa7267da640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562f2d6c4800 0x562f2d614c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7267da640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562f2d6c4400 0x562f2d615180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7277dc640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562f2d6c4400 0x562f2d615180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7277dc640 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562f2d6c4400 0x562f2d615180 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.275+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4800 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.276+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4800 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.277+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.278+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.279+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4800 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.280+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4800 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c5400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2e129c00 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e128800 con 0x562f2d6c4800 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.281+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4c00 con 0x562f2e129c00 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5000 con 0x562f2e129c00 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5800 con 0x562f2d6c5400 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 zone default found 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 4 Realm: () 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 zonegroup default 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5c00 con 0x562f2d6c4400 2024-04-27T05:21:35.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.282+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114000 con 0x562f2d6c5400 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114400 con 0x562f2d6c5400 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 20 started sync module instance, tier type = 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.283+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114800 con 0x562f2d6c4800 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.284+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.284+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94760624761088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114c00 con 0x562f2d6c4800 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.284+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e115000 con 0x562f2e129c00 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.285+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94760624761088] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.285+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.285+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94760624762240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e115400 con 0x562f2e129c00 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.285+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e115800 con 0x562f2d6c5400 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.286+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.286+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94760624763392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114000 con 0x562f2d6c5400 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.286+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114400 con 0x562f2d6c4800 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.286+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94760624762240] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.287+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.287+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94760624763392] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.287+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94760624764544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e115800 con 0x562f2d6c4800 2024-04-27T05:21:35.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.287+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5800 con 0x562f2e129c00 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.288+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.288+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94760624765696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c4c00 con 0x562f2e129c00 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.288+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5000 con 0x562f2d6c4400 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.288+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94760624764544] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.289+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94760624765696] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.289+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.289+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94760624766848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d6c5c00 con 0x562f2d6c4400 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.289+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e286c00 con 0x562f2d6c4400 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.290+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94760624766848] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.290+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.290+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94760624768000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114800 con 0x562f2d6c4400 2024-04-27T05:21:35.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.290+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114c00 con 0x562f2d6c4400 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94760624768000] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94760624769152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e114400 con 0x562f2d6c4400 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=0 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=1 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=2 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=3 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=4 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=5 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.291+0000 7fa72aa11880 20 add_watcher() i=6 2024-04-27T05:21:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.292+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94760624769152] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.292+0000 7fa72aa11880 20 add_watcher() i=7 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.292+0000 7fa72aa11880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.292+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e115800 con 0x562f2e129c00 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.292+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.293+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94760624770304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2e115000 con 0x562f2e129c00 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.293+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94760624770304] v29'82 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa70dfa9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa72aa11880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa72aa11880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa72aa11880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa72aa11880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f2d3548c0 con 0x562f2e115c00 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa7267da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa7267da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43172/0 (socket says 172.21.15.138:43172) 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa7267da640 1 -- 172.21.15.138:0/1090126310 learned_addr learned my addr 172.21.15.138:0/1090126310 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.294+0000 7fa7267da640 1 -- 172.21.15.138:0/1090126310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d3a2680 con 0x562f2e115c00 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa7267da640 1 --2- 172.21.15.138:0/1090126310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x562f2d6a2b70 tx=0x562f2d6880c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d2ee1a88d0bc39 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa70d7a8640 1 -- 172.21.15.138:0/1090126310 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2e116580 con 0x562f2e115c00 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa70d7a8640 1 -- 172.21.15.138:0/1090126310 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2e202000 con 0x562f2e115c00 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa70d7a8640 1 -- 172.21.15.138:0/1090126310 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2e1166e0 con 0x562f2e115c00 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 -- 172.21.15.138:0/1090126310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 msgr2=0x562f2d614680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 --2- 172.21.15.138:0/1090126310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x562f2d6a2b70 tx=0x562f2d6880c0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 -- 172.21.15.138:0/1090126310 shutdown_connections 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 --2- 172.21.15.138:0/1090126310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 -- 172.21.15.138:0/1090126310 >> 172.21.15.138:0/1090126310 conn(0x562f2e115400 msgr2=0x562f2d43b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 -- 172.21.15.138:0/1090126310 shutdown_connections 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.295+0000 7fa72aa11880 1 -- 172.21.15.138:0/1090126310 wait complete. 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa72aa11880 1 Processor -- start 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa72aa11880 1 -- start start 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa72aa11880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa72aa11880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f2d354780 con 0x562f2e115c00 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa7267da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa7267da640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43176/0 (socket says 172.21.15.138:43176) 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa7267da640 1 -- 172.21.15.138:0/835745083 learned_addr learned my addr 172.21.15.138:0/835745083 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa7267da640 1 -- 172.21.15.138:0/835745083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f2d3a24e0 con 0x562f2e115c00 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa7267da640 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x562f2d6a3560 tx=0x562f2d688000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa70bfa5640 1 -- 172.21.15.138:0/835745083 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2e116840 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa70bfa5640 1 -- 172.21.15.138:0/835745083 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f2e202780 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562f2d334ea0 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.296+0000 7fa70bfa5640 1 -- 172.21.15.138:0/835745083 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562f2e116580 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.297+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f2d3356c0 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.297+0000 7fa70bfa5640 1 -- 172.21.15.138:0/835745083 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562f2d35ab00 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.297+0000 7fa70bfa5640 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e114800 0x562f2d615700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.297+0000 7fa70bfa5640 1 -- 172.21.15.138:0/835745083 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x562f2d343dc0 con 0x562f2e115c00 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.297+0000 7fa72aa11880 5 note: GC not initialized 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.297+0000 7fa7277dc640 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e114800 0x562f2d615700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa7097a0640 20 reqs_thread_entry: start 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa72aa11880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa72aa11880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa7277dc640 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e114800 0x562f2d615700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562f2e2ae9c0 tx=0x562f2d688960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa70779c640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:35.321 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa706f7a640 20 reqs_thread_entry: start 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa72aa11880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a4000 con 0x562f2e129c00 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.298+0000 7fa72aa11880 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo1", 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.326 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.299+0000 7fa70dfa9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.299+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94760624770304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a4400 con 0x562f2e129c00 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.300+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94760624770304] v29'83 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.300+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94760624761088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a4800 con 0x562f2d6c4800 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.301+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94760624761088] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.301+0000 7fa72aa11880 20 remove_watcher() i=0 2024-04-27T05:21:35.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.301+0000 7fa72aa11880 2 removed watcher, disabling cache 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.301+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94760624762240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a4c00 con 0x562f2e129c00 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.302+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94760624762240] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.302+0000 7fa72aa11880 20 remove_watcher() i=1 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.302+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94760624763392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a5000 con 0x562f2d6c5400 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.303+0000 7fa726fdb640 1 -- 172.21.15.138:0/2525780728 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94760624763392] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6fc0 con 0x562f2d6c5400 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.303+0000 7fa72aa11880 20 remove_watcher() i=2 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.303+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94760624764544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a5400 con 0x562f2d6c4800 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.304+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94760624764544] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d7200 con 0x562f2d6c4800 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.304+0000 7fa72aa11880 20 remove_watcher() i=3 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.304+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94760624765696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a5800 con 0x562f2e129c00 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.305+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94760624765696] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2e129c00 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.305+0000 7fa72aa11880 20 remove_watcher() i=4 2024-04-27T05:21:35.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.305+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94760624766848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a5c00 con 0x562f2d6c4400 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.306+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94760624766848] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.306+0000 7fa72aa11880 20 remove_watcher() i=5 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.306+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94760624768000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a4000 con 0x562f2d6c4400 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94760624768000] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 20 remove_watcher() i=6 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94760624769152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562f2d7a4400 con 0x562f2d6c4400 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa7277dc640 1 -- 172.21.15.138:0/2525780728 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94760624769152] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562f2d6d6900 con 0x562f2d6c4400 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 20 remove_watcher() i=7 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562f2d6c4400 msgr2=0x562f2d615180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562f2d6c4400 0x562f2d615180 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562f2e129c00 msgr2=0x562f2d613b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562f2e129c00 0x562f2d613b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.307+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562f2d6c4800 msgr2=0x562f2d614c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562f2d6c4800 0x562f2d614c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562f2d6c5400 msgr2=0x562f2d614100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562f2d6c5400 0x562f2d614100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2d6c4000 msgr2=0x562f2d613600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2d6c4000 0x562f2d613600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x562f2d43cc00 tx=0x562f2d688120 comp rx=0 tx=0).stop 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 msgr2=0x562f2d612b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/2525780728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e129800 0x562f2d612b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x562f2e1ed920 tx=0x562f2d443a40 comp rx=0 tx=0).stop 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 reap_dead start 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 shutdown_connections 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 >> 172.21.15.138:0/2525780728 conn(0x562f2d331400 msgr2=0x562f2d43b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa7267da640 1 -- 172.21.15.138:0/2525780728 reap_dead start 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 shutdown_connections 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/2525780728 wait complete. 2024-04-27T05:21:35.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e114800 msgr2=0x562f2d615700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e114800 0x562f2d615700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562f2e2ae9c0 tx=0x562f2d688960 comp rx=0 tx=0).stop 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 msgr2=0x562f2d614680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x562f2d6a3560 tx=0x562f2d688000 comp rx=0 tx=0).stop 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 shutdown_connections 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e114800 0x562f2d615700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 --2- 172.21.15.138:0/835745083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2e115c00 0x562f2d614680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 >> 172.21.15.138:0/835745083 conn(0x562f2e115400 msgr2=0x562f2d43b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 shutdown_connections 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.308+0000 7fa72aa11880 1 -- 172.21.15.138:0/835745083 wait complete. 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e286000 msgr2=0x562f2d613080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e286000 0x562f2d613080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x562f2e234510 tx=0x562f2d4434a0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 msgr2=0x562f2d612580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x562f2e1ecb40 tx=0x562f2d442c60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 shutdown_connections 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562f2e286000 0x562f2d613080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 --2- 172.21.15.138:0/3720239890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562f2d331000 0x562f2d612580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 >> 172.21.15.138:0/3720239890 conn(0x562f2d330c00 msgr2=0x562f2d439000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 shutdown_connections 2024-04-27T05:21:35.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.309+0000 7fa72aa11880 1 -- 172.21.15.138:0/3720239890 wait complete. 2024-04-27T05:21:35.337 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.338 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-27T05:21:35.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-27T05:21:35.338 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-04-27T05:21:35.418 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.418 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a10d27880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5626d4b13400 0x5626d4df4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a10d27880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5626d4dea600 con 0x5626d4b13000 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a10d27880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5626d4b52280 con 0x5626d4b13400 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a0cecd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5626d4b13400 0x5626d4df4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a0d6ce640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5626d4b13000 0x5626d4c1a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60830/0 (socket says 172.21.15.138:60830) 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a0cecd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5626d4b13400 0x5626d4df4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43192/0 (socket says 172.21.15.138:43192) 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/120693507 learned_addr learned my addr 172.21.15.138:0/120693507 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a0cecd640 1 -- 172.21.15.138:0/120693507 >> v1:172.21.15.114:6789/0 conn(0x5626d4b13000 legacy=0x5626d4c1a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.417+0000 7f4a0cecd640 1 -- 172.21.15.138:0/120693507 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b17380 con 0x5626d4b13400 2024-04-27T05:21:35.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.419+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/120693507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13400 0x5626d4df4580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5626d4c1ec00 tx=0x5626d4c24de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecde68fc482c410 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.419+0000 7f4a0c6cc640 1 -- 172.21.15.138:0/120693507 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2bce0 con 0x5626d4b13400 2024-04-27T05:21:35.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.419+0000 7f4a0c6cc640 1 -- 172.21.15.138:0/120693507 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d4debb00 con 0x5626d4b13400 2024-04-27T05:21:35.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.419+0000 7f4a0c6cc640 1 -- 172.21.15.138:0/120693507 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2be40 con 0x5626d4b13400 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- 172.21.15.138:0/120693507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13400 msgr2=0x5626d4df4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 --2- 172.21.15.138:0/120693507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13400 0x5626d4df4580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5626d4c1ec00 tx=0x5626d4c24de0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- 172.21.15.138:0/120693507 shutdown_connections 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 --2- 172.21.15.138:0/120693507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13400 0x5626d4df4580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- 172.21.15.138:0/120693507 >> 172.21.15.138:0/120693507 conn(0x5626d4b12c00 msgr2=0x5626d4c1a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- 172.21.15.138:0/120693507 shutdown_connections 2024-04-27T05:21:35.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- 172.21.15.138:0/120693507 wait complete. 2024-04-27T05:21:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.420+0000 7f4a10d27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d4b52780 con 0x5626d4b13000 2024-04-27T05:21:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0d6ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0d6ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43194/0 (socket says 172.21.15.138:43194) 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/1476460036 learned_addr learned my addr 172.21.15.138:0/1476460036 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/1476460036 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b16ea0 con 0x5626d4b13000 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/1476460036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5626d59cecc0 tx=0x5626d4c24cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9002ad3f7f6ff21e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0becb640 1 -- 172.21.15.138:0/1476460036 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2b4a0 con 0x5626d4b13000 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0becb640 1 -- 172.21.15.138:0/1476460036 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d4debe00 con 0x5626d4b13000 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.421+0000 7f4a0becb640 1 -- 172.21.15.138:0/1476460036 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2bce0 con 0x5626d4b13000 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- 172.21.15.138:0/1476460036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 msgr2=0x5626d4df4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 --2- 172.21.15.138:0/1476460036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5626d59cecc0 tx=0x5626d4c24cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- 172.21.15.138:0/1476460036 shutdown_connections 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 --2- 172.21.15.138:0/1476460036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- 172.21.15.138:0/1476460036 >> 172.21.15.138:0/1476460036 conn(0x5626d4b12c00 msgr2=0x5626d4c1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- 172.21.15.138:0/1476460036 shutdown_connections 2024-04-27T05:21:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- 172.21.15.138:0/1476460036 wait complete. 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a10d27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d4b528c0 con 0x5626d4b13000 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a0d6ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a0d6ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43196/0 (socket says 172.21.15.138:43196) 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/4076378838 learned_addr learned my addr 172.21.15.138:0/4076378838 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.422+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/4076378838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b176c0 con 0x5626d4b13000 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x5626d59ceea0 tx=0x5626d4c24c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a0a6c8640 1 -- 172.21.15.138:0/4076378838 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2be40 con 0x5626d4b13000 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a0a6c8640 1 -- 172.21.15.138:0/4076378838 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d59e4000 con 0x5626d4b13000 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626d4b171e0 con 0x5626d4b13000 2024-04-27T05:21:35.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a0a6c8640 1 -- 172.21.15.138:0/4076378838 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2b4a0 con 0x5626d4b13000 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626d4b84ea0 con 0x5626d4b13000 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a0a6c8640 1 -- 172.21.15.138:0/4076378838 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5626d4b3d600 con 0x5626d4b13000 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.423+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a10d27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a10d27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d4b36280 con 0x5626d5905800 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0a6c8640 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d5a6a000 0x5626d4df5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0a6c8640 1 -- 172.21.15.138:0/4076378838 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5626d4b25a40 con 0x5626d4b13000 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d5a6a000 0x5626d4df5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0decf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0decf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43200/0 (socket says 172.21.15.138:43200) 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0decf640 1 -- 172.21.15.138:0/3693568656 learned_addr learned my addr 172.21.15.138:0/3693568656 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d5a6a000 0x5626d4df5080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5626d5a1a450 tx=0x5626d4c254a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0decf640 1 -- 172.21.15.138:0/3693568656 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b84d00 con 0x5626d5905800 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a0decf640 1 --2- 172.21.15.138:0/3693568656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5626d59cfb30 tx=0x5626d4c25c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a6882539d25974 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a07ec3640 1 -- 172.21.15.138:0/3693568656 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2be40 con 0x5626d5905800 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a07ec3640 1 -- 172.21.15.138:0/3693568656 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d59e4300 con 0x5626d5905800 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.424+0000 7f4a07ec3640 1 -- 172.21.15.138:0/3693568656 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2b4a0 con 0x5626d5905800 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- 172.21.15.138:0/3693568656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 msgr2=0x5626d4df4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 --2- 172.21.15.138:0/3693568656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5626d59cfb30 tx=0x5626d4c25c20 comp rx=0 tx=0).stop 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- 172.21.15.138:0/3693568656 shutdown_connections 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 --2- 172.21.15.138:0/3693568656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- 172.21.15.138:0/3693568656 >> 172.21.15.138:0/3693568656 conn(0x5626d4b13400 msgr2=0x5626d4c1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- 172.21.15.138:0/3693568656 shutdown_connections 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- 172.21.15.138:0/3693568656 wait complete. 2024-04-27T05:21:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a10d27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d4b37180 con 0x5626d5905800 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a0decf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a0decf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43204/0 (socket says 172.21.15.138:43204) 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.425+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 learned_addr learned my addr 172.21.15.138:0/2886077493 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b84b60 con 0x5626d5905800 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5626d5911470 tx=0x5626d4c25a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a066c0640 1 -- 172.21.15.138:0/2886077493 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2be40 con 0x5626d5905800 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a066c0640 1 -- 172.21.15.138:0/2886077493 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d59e4480 con 0x5626d5905800 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626d4b849c0 con 0x5626d5905800 2024-04-27T05:21:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a066c0640 1 -- 172.21.15.138:0/2886077493 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4b2b4a0 con 0x5626d5905800 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.426+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626d4b84820 con 0x5626d5905800 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a066c0640 1 -- 172.21.15.138:0/2886077493 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5626d4b3d8c0 con 0x5626d5905800 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a066c0640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4e9e000 0x5626d4df5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a066c0640 1 -- 172.21.15.138:0/2886077493 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5626d4b25c00 con 0x5626d5905800 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4e9e000 0x5626d4df5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5626d5905c00 0x5626d4df5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9e400 con 0x5626d5905c00 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.427+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5626d5905c00 0x5626d4df5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4e9e000 0x5626d4df5600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5626d4c1ec00 tx=0x5626d4e66120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5626d5905c00 0x5626d4df5b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 realm 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9e800 con 0x5626d5905c00 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9ec00 con 0x5626d5905c00 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.428+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9f000 con 0x5626d5905c00 2024-04-27T05:21:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5626d4e9f400 0x5626d4df6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9e400 con 0x5626d4e9f400 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5626d4e9f400 0x5626d4df6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5626d4e9f400 0x5626d4df6100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 20 searching for the correct realm 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.429+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9e800 con 0x5626d4e9f400 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0d6ce640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5626d4e9e800 0x5626d4df6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9e400 con 0x5626d4e9e800 2024-04-27T05:21:35.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5626d4e9e800 0x5626d4df6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5626d4e9e800 0x5626d4df6c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5626d4e9e400 0x5626d4df7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.430+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.431+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5626d4e9e400 0x5626d4df7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.431+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5626d4e9e400 0x5626d4df7180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.431+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.431+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.431+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.431+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e800 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.432+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e800 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.433+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.434+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.435+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.435+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.435+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.435+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.435+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.435+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e800 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.436+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e800 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.437+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9f400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d5905c00 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5904800 con 0x5626d4e9e800 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a10d27880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a10d27880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a10d27880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a10d27880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.438+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9ec00 con 0x5626d5905c00 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9f000 con 0x5626d5905c00 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9f800 con 0x5626d4e9f400 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 zone default found 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 4 Realm: () 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 zonegroup default 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.439+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9fc00 con 0x5626d4e9e400 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2000 con 0x5626d4e9f400 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2400 con 0x5626d4e9f400 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 started sync module instance, tier type = 2024-04-27T05:21:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.440+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2800 con 0x5626d4e9e800 2024-04-27T05:21:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.441+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.441+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94724780116224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2c00 con 0x5626d4e9e800 2024-04-27T05:21:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.441+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec3000 con 0x5626d5905c00 2024-04-27T05:21:35.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.442+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94724780116224] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.443+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.443+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94724780117376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec3400 con 0x5626d5905c00 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.443+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec3800 con 0x5626d4e9f400 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.444+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94724780117376] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.444+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.444+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94724780118528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2000 con 0x5626d4e9f400 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.444+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2400 con 0x5626d4e9e800 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.445+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.445+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94724780119680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec3800 con 0x5626d4e9e800 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.445+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9f800 con 0x5626d5905c00 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.445+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94724780118528] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.446+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.446+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94724780120832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9ec00 con 0x5626d5905c00 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.446+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9f000 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.446+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94724780119680] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.447+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.447+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94724780120832] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.447+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94724780121984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4e9fc00 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.447+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d5a6ac00 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.448+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.448+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94724780123136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2800 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.448+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2c00 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.448+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94724780121984] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94724780123136] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94724780124288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec2400 con 0x5626d4e9e400 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=0 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=1 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=2 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=3 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=4 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=5 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.449+0000 7f4a10d27880 20 add_watcher() i=6 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.450+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94724780124288] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.450+0000 7f4a10d27880 20 add_watcher() i=7 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.450+0000 7f4a10d27880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.450+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec3800 con 0x5626d5905c00 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.451+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.451+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94724780125440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4ec3000 con 0x5626d5905c00 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.451+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94724780125440] v29'84 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.451+0000 7f49f469c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.452+0000 7f4a10d27880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.452+0000 7f4a10d27880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.452+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.452+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.452+0000 7f4a10d27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.452+0000 7f4a10d27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d4b36780 con 0x5626d4ec3c00 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.453+0000 7f4a0cecd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.453+0000 7f4a0cecd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43212/0 (socket says 172.21.15.138:43212) 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.453+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2520455367 learned_addr learned my addr 172.21.15.138:0/2520455367 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.455+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2520455367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b84680 con 0x5626d4ec3c00 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.455+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2520455367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5626d5a907e0 tx=0x5626d4e660c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c825f582c79e1d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.455+0000 7f49f3e9b640 1 -- 172.21.15.138:0/2520455367 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4ec4580 con 0x5626d4ec3c00 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.455+0000 7f49f3e9b640 1 -- 172.21.15.138:0/2520455367 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d59e4000 con 0x5626d4ec3c00 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.455+0000 7f49f3e9b640 1 -- 172.21.15.138:0/2520455367 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4ec46e0 con 0x5626d4ec3c00 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- 172.21.15.138:0/2520455367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 msgr2=0x5626d4df6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2520455367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5626d5a907e0 tx=0x5626d4e660c0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- 172.21.15.138:0/2520455367 shutdown_connections 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2520455367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- 172.21.15.138:0/2520455367 >> 172.21.15.138:0/2520455367 conn(0x5626d4ec3400 msgr2=0x5626d4c1d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- 172.21.15.138:0/2520455367 shutdown_connections 2024-04-27T05:21:35.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- 172.21.15.138:0/2520455367 wait complete. 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 Processor -- start 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- start start 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.456+0000 7f4a10d27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d4b36500 con 0x5626d4ec3c00 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a0cecd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a0cecd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43228/0 (socket says 172.21.15.138:43228) 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2992346146 learned_addr learned my addr 172.21.15.138:0/2992346146 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2992346146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d4b844e0 con 0x5626d4ec3c00 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a0cecd640 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5626d4e81740 tx=0x5626d4e66000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f49f2698640 1 -- 172.21.15.138:0/2992346146 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4ec4840 con 0x5626d4ec3c00 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f49f2698640 1 -- 172.21.15.138:0/2992346146 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626d59e4780 con 0x5626d4ec3c00 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626d4b16ea0 con 0x5626d4ec3c00 2024-04-27T05:21:35.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f49f2698640 1 -- 172.21.15.138:0/2992346146 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5626d4ec4580 con 0x5626d4ec3c00 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.457+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626d4b176c0 con 0x5626d4ec3c00 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f49f2698640 1 -- 172.21.15.138:0/2992346146 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5626d4b3cb00 con 0x5626d4ec3c00 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f49f2698640 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4ec2800 0x5626d4df7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f49f2698640 1 -- 172.21.15.138:0/2992346146 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5626d4b25dc0 con 0x5626d4ec3c00 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f4a10d27880 5 note: GC not initialized 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4ec2800 0x5626d4df7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f49efe93640 20 reqs_thread_entry: start 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f4a10d27880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.458+0000 7f4a0decf640 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4ec2800 0x5626d4df7700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5626d5a90840 tx=0x5626d4e66960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f49ede8f640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f49ed66d640 20 reqs_thread_entry: start 2024-04-27T05:21:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f82000 con 0x5626d5905c00 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f82400 con 0x5626d4e9e800 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-04-27T05:21:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.459+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~621 in=621b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f82c00 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~621] v29'8 uv8 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a10d27880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a10d27880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a10d27880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94724780127744 in=809b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f83000 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94724780120832 notify 124554051591 ret 0) v3 ==== 839+0+0 (crc 0 0 0) 0x5626d4e65d40 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a07ec3640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051591 cookie 94724780120832 notifier 4413 bl.length()=797 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(65 notify.4 [notify cookie 94724780127744 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a07ec3640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a07ec3640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a07ec3640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f94000 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94724780127744 notify 124554051591 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5626d4f09860 con 0x5626d5905c00 2024-04-27T05:21:35.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.461+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f83400 con 0x5626d4e9e800 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a10d27880 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a10d27880 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a10d27880 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94724780127744 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f83800 con 0x5626d4e9e800 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== watch-notify(notify (1) cookie 94724780116224 notify 124554051587 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x5626d4f09ba0 con 0x5626d4e9e800 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a07ec3640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051587 cookie 94724780116224 notifier 4413 bl.length()=166 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a07ec3640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a07ec3640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a07ec3640 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f94400 con 0x5626d4e9e800 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(68 notify.0 [notify cookie 94724780127744 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.463+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== watch-notify(notify_complete (2) cookie 94724780127744 notify 124554051587 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5626d4f09d40 con 0x5626d4e9e800 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:35.483 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo2", 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "" 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.484 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo1", 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo2", 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.485 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.487 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.487 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.464+0000 7f49f469c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.464+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94724780125440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f83c00 con 0x5626d5905c00 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.465+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94724780125440] v29'85 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.465+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94724780116224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f94000 con 0x5626d4e9e800 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.466+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94724780116224] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.466+0000 7f4a10d27880 20 remove_watcher() i=0 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.466+0000 7f4a10d27880 2 removed watcher, disabling cache 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.466+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94724780117376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f83000 con 0x5626d5905c00 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.467+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94724780117376] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.467+0000 7f4a10d27880 20 remove_watcher() i=1 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.467+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94724780118528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f94800 con 0x5626d4e9f400 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.468+0000 7f4a0d6ce640 1 -- 172.21.15.138:0/2886077493 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94724780118528] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0fc0 con 0x5626d4e9f400 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.468+0000 7f4a10d27880 20 remove_watcher() i=2 2024-04-27T05:21:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.468+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94724780119680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f94c00 con 0x5626d4e9e800 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.468+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94724780119680] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb1200 con 0x5626d4e9e800 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.468+0000 7f4a10d27880 20 remove_watcher() i=3 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.468+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94724780120832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f95000 con 0x5626d5905c00 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.469+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94724780120832] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d5905c00 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.470+0000 7f4a10d27880 20 remove_watcher() i=4 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.470+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94724780121984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f95400 con 0x5626d4e9e400 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.470+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94724780121984] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.470+0000 7f4a10d27880 20 remove_watcher() i=5 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.470+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94724780123136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f82000 con 0x5626d4e9e400 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.471+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94724780123136] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.471+0000 7f4a10d27880 20 remove_watcher() i=6 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.471+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94724780124288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5626d4f82c00 con 0x5626d4e9e400 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a0decf640 1 -- 172.21.15.138:0/2886077493 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94724780124288] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5626d4eb0900 con 0x5626d4e9e400 2024-04-27T05:21:35.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 20 remove_watcher() i=7 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5626d4e9e400 msgr2=0x5626d4df7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5626d4e9e400 0x5626d4df7180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5626d5905c00 msgr2=0x5626d4df5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5626d5905c00 0x5626d4df5b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5626d4e9e800 msgr2=0x5626d4df6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5626d4e9e800 0x5626d4df6c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5626d4e9f400 msgr2=0x5626d4df6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5626d4e9f400 0x5626d4df6100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4e9e000 msgr2=0x5626d4df5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.472+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4e9e000 0x5626d4df5600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5626d4c1ec00 tx=0x5626d4e66120 comp rx=0 tx=0).stop 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 msgr2=0x5626d4df4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2886077493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d5905800 0x5626d4df4b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5626d5911470 tx=0x5626d4c25a40 comp rx=0 tx=0).stop 2024-04-27T05:21:35.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 reap_dead start 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a0cecd640 1 -- 172.21.15.138:0/2886077493 reap_dead start 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 shutdown_connections 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 >> 172.21.15.138:0/2886077493 conn(0x5626d4b13400 msgr2=0x5626d4c1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 shutdown_connections 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2886077493 wait complete. 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4ec2800 msgr2=0x5626d4df7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4ec2800 0x5626d4df7700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5626d5a90840 tx=0x5626d4e66960 comp rx=0 tx=0).stop 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 msgr2=0x5626d4df6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5626d4e81740 tx=0x5626d4e66000 comp rx=0 tx=0).stop 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 shutdown_connections 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d4ec2800 0x5626d4df7700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/2992346146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4ec3c00 0x5626d4df6680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 >> 172.21.15.138:0/2992346146 conn(0x5626d4ec3400 msgr2=0x5626d4c1d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 shutdown_connections 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/2992346146 wait complete. 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d5a6a000 msgr2=0x5626d4df5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d5a6a000 0x5626d4df5080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5626d5a1a450 tx=0x5626d4c254a0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 msgr2=0x5626d4df4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x5626d59ceea0 tx=0x5626d4c24c60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 shutdown_connections 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5626d5a6a000 0x5626d4df5080 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 --2- 172.21.15.138:0/4076378838 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5626d4b13000 0x5626d4df4580 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.473+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 >> 172.21.15.138:0/4076378838 conn(0x5626d4b12c00 msgr2=0x5626d4c1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.474+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 shutdown_connections 2024-04-27T05:21:35.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.474+0000 7f4a10d27880 1 -- 172.21.15.138:0/4076378838 wait complete. 2024-04-27T05:21:35.497 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.498 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:21:35.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:21:35.498 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:21:35.540 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.540 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.538+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.538+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e7e98880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x559836397000 0x559836678580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e7e98880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55983666e600 con 0x559836397400 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e7e98880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5598363d6280 con 0x559836397000 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e664b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x559836397000 0x559836678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e5e4a640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x559836397400 0x55983649e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60846/0 (socket says 172.21.15.138:60846) 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e664b640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x559836397000 0x559836678580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43230/0 (socket says 172.21.15.138:43230) 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e664b640 1 -- 172.21.15.138:0/815679624 learned_addr learned my addr 172.21.15.138:0/815679624 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e664b640 1 -- 172.21.15.138:0/815679624 >> v1:172.21.15.114:6789/0 conn(0x559836397400 legacy=0x55983649e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.539+0000 7f17e664b640 1 -- 172.21.15.138:0/815679624 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983639b380 con 0x559836397000 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e664b640 1 --2- 172.21.15.138:0/815679624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5598364a2d50 tx=0x5598364a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b8f04d986eca5c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e5649640 1 -- 172.21.15.138:0/815679624 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363afce0 con 0x559836397000 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e5649640 1 -- 172.21.15.138:0/815679624 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55983666fb00 con 0x559836397000 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e5649640 1 -- 172.21.15.138:0/815679624 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363afe40 con 0x559836397000 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 -- 172.21.15.138:0/815679624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 msgr2=0x559836678580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 --2- 172.21.15.138:0/815679624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5598364a2d50 tx=0x5598364a8de0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 -- 172.21.15.138:0/815679624 shutdown_connections 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 --2- 172.21.15.138:0/815679624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 -- 172.21.15.138:0/815679624 >> 172.21.15.138:0/815679624 conn(0x559836396c00 msgr2=0x55983649e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 -- 172.21.15.138:0/815679624 shutdown_connections 2024-04-27T05:21:35.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.541+0000 7f17e7e98880 1 -- 172.21.15.138:0/815679624 wait complete. 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e7e98880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e7e98880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363d6780 con 0x559836397000 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e664b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e664b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43246/0 (socket says 172.21.15.138:43246) 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e664b640 1 -- 172.21.15.138:0/771182188 learned_addr learned my addr 172.21.15.138:0/771182188 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.542+0000 7f17e664b640 1 -- 172.21.15.138:0/771182188 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983639aea0 con 0x559836397000 2024-04-27T05:21:35.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e664b640 1 --2- 172.21.15.138:0/771182188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55983719b560 tx=0x5598364a8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d810e7e31ffa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e4e48640 1 -- 172.21.15.138:0/771182188 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363af4a0 con 0x559836397000 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e4e48640 1 -- 172.21.15.138:0/771182188 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55983666fe00 con 0x559836397000 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e4e48640 1 -- 172.21.15.138:0/771182188 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363afce0 con 0x559836397000 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 -- 172.21.15.138:0/771182188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 msgr2=0x559836678580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 --2- 172.21.15.138:0/771182188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55983719b560 tx=0x5598364a8cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 -- 172.21.15.138:0/771182188 shutdown_connections 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 --2- 172.21.15.138:0/771182188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 -- 172.21.15.138:0/771182188 >> 172.21.15.138:0/771182188 conn(0x559836396c00 msgr2=0x55983649f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 -- 172.21.15.138:0/771182188 shutdown_connections 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 -- 172.21.15.138:0/771182188 wait complete. 2024-04-27T05:21:35.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.543+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e7e98880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e7e98880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363d68c0 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e664b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e664b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43250/0 (socket says 172.21.15.138:43250) 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e664b640 1 -- 172.21.15.138:0/693963953 learned_addr learned my addr 172.21.15.138:0/693963953 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e664b640 1 -- 172.21.15.138:0/693963953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983639b6c0 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e664b640 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5598372532c0 tx=0x5598364a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e3645640 1 -- 172.21.15.138:0/693963953 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363afe40 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e3645640 1 -- 172.21.15.138:0/693963953 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559837176000 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55983639b1e0 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e3645640 1 -- 172.21.15.138:0/693963953 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363af4a0 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.544+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559836408ea0 con 0x559836397000 2024-04-27T05:21:35.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e3645640 1 -- 172.21.15.138:0/693963953 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5598363c0b00 con 0x559836397000 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e7e98880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e7e98880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363d6c80 con 0x55983718ec00 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e3645640 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55983718fc00 0x559836679080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e3645640 1 -- 172.21.15.138:0/693963953 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5598363a9a40 con 0x559836397000 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e664b640 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55983718fc00 0x559836679080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e6e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e6e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43266/0 (socket says 172.21.15.138:43266) 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.545+0000 7f17e6e4c640 1 -- 172.21.15.138:0/4266288419 learned_addr learned my addr 172.21.15.138:0/4266288419 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e6e4c640 1 -- 172.21.15.138:0/4266288419 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559836408d00 con 0x55983718ec00 2024-04-27T05:21:35.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e664b640 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55983718fc00 0x559836679080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x559837298180 tx=0x5598364a8cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/4266288419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5598372535f0 tx=0x5598364a9c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54252e01594b6dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e0e40640 1 -- 172.21.15.138:0/4266288419 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363afe40 con 0x55983718ec00 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e0e40640 1 -- 172.21.15.138:0/4266288419 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559837176600 con 0x55983718ec00 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e0e40640 1 -- 172.21.15.138:0/4266288419 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363af4a0 con 0x55983718ec00 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 -- 172.21.15.138:0/4266288419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 msgr2=0x559836678b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 --2- 172.21.15.138:0/4266288419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x5598372535f0 tx=0x5598364a9c20 comp rx=0 tx=0).stop 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 -- 172.21.15.138:0/4266288419 shutdown_connections 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 --2- 172.21.15.138:0/4266288419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 -- 172.21.15.138:0/4266288419 >> 172.21.15.138:0/4266288419 conn(0x559836397400 msgr2=0x5598364a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 -- 172.21.15.138:0/4266288419 shutdown_connections 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.546+0000 7f17e7e98880 1 -- 172.21.15.138:0/4266288419 wait complete. 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e7e98880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e7e98880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363ba280 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e6e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e6e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43282/0 (socket says 172.21.15.138:43282) 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 learned_addr learned my addr 172.21.15.138:0/174706570 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559836408b60 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5598372533b0 tx=0x5598364a9980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17df63d640 1 -- 172.21.15.138:0/174706570 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363afe40 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17df63d640 1 -- 172.21.15.138:0/174706570 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559837176780 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17df63d640 1 -- 172.21.15.138:0/174706570 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598363af4a0 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598364089c0 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.547+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559836408820 con 0x55983718ec00 2024-04-27T05:21:35.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17df63d640 1 -- 172.21.15.138:0/174706570 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5598363c1600 con 0x55983718ec00 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17df63d640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559836714000 0x559836679600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17df63d640 1 -- 172.21.15.138:0/174706570 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5598363a9c00 con 0x55983718ec00 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559836714000 0x559836679600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55983718f000 0x559836679b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.548+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983718f400 con 0x55983718f000 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55983718f000 0x559836679b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559836714000 0x559836679600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5598364a3380 tx=0x5598366da120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55983718f000 0x559836679b80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 realm 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983718f800 con 0x55983718f000 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715000 con 0x55983718f000 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.549+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715400 con 0x55983718f000 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559836715800 0x55983667a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715c00 con 0x559836715800 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e664b640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559836715800 0x55983667a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e664b640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559836715800 0x55983667a100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 20 searching for the correct realm 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.550+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715000 con 0x559836715800 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e664b640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559836715000 0x55983667ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715c00 con 0x559836715000 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559836715000 0x55983667ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559836715000 0x55983667ac00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559836715c00 0x55983667b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.551+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.552+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559836715c00 0x55983667b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.552+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559836715c00 0x55983667b180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.552+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.552+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.552+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.552+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715000 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.553+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715000 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.554+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.555+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715000 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.556+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.557+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.557+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.557+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.557+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.557+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.557+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715000 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715800 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715c00 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x55983718f000 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.558+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836714c00 con 0x559836715000 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715400 con 0x55983718f000 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983718f400 con 0x55983718f000 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983718f800 con 0x559836715800 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 zone default found 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 4 Realm: () 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 zonegroup default 2024-04-27T05:21:35.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.559+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e000 con 0x559836715c00 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e400 con 0x559836715800 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e800 con 0x559836715800 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 started sync module instance, tier type = 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.560+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673ec00 con 0x559836715000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.561+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.561+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94112236112128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673f000 con 0x559836715000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.561+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673f400 con 0x55983718f000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.562+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94112236112128] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.562+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.562+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94112236113280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673f800 con 0x55983718f000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.562+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673fc00 con 0x559836715800 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.563+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.563+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94112236114432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e400 con 0x559836715800 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.563+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e800 con 0x559836715000 2024-04-27T05:21:35.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.563+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94112236113280] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.564+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94112236114432] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.564+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.564+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94112236115584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673fc00 con 0x559836715000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.564+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e000 con 0x55983718f000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.565+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.565+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94112236116736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983718f800 con 0x55983718f000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.565+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983718f400 con 0x559836715c00 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.565+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94112236115584] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.566+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.566+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94112236117888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836715400 con 0x559836715c00 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.566+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5598372fac00 con 0x559836715c00 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.566+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94112236116736] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.567+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94112236117888] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.567+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.567+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94112236119040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5598372fa800 con 0x559836715c00 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.567+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673ec00 con 0x559836715c00 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94112236119040] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94112236120192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673f000 con 0x559836715c00 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=0 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=1 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=2 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=3 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=4 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=5 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.568+0000 7f17e7e98880 20 add_watcher() i=6 2024-04-27T05:21:35.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.569+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94112236120192] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.569+0000 7f17e7e98880 20 add_watcher() i=7 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.569+0000 7f17e7e98880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.569+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673e800 con 0x55983718f000 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.569+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.569+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94112236121344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55983673f400 con 0x55983718f000 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.570+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94112236121344] v29'86 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.570+0000 7f17cd619640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.570+0000 7f17e7e98880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.570+0000 7f17e7e98880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.570+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.570+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e7e98880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e7e98880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363ba8c0 con 0x559836714c00 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e5e4a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e5e4a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43298/0 (socket says 172.21.15.138:43298) 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e5e4a640 1 -- 172.21.15.138:0/3394757606 learned_addr learned my addr 172.21.15.138:0/3394757606 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e5e4a640 1 -- 172.21.15.138:0/3394757606 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559836408680 con 0x559836714c00 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/3394757606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5598366f6ae0 tx=0x5598366da180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7432d90c9bc5e28c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.571+0000 7f17cce18640 1 -- 172.21.15.138:0/3394757606 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559836740580 con 0x559836714c00 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17cce18640 1 -- 172.21.15.138:0/3394757606 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559837176000 con 0x559836714c00 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17cce18640 1 -- 172.21.15.138:0/3394757606 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5598367406e0 con 0x559836714c00 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 -- 172.21.15.138:0/3394757606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 msgr2=0x55983667a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 --2- 172.21.15.138:0/3394757606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5598366f6ae0 tx=0x5598366da180 comp rx=0 tx=0).stop 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 -- 172.21.15.138:0/3394757606 shutdown_connections 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 --2- 172.21.15.138:0/3394757606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 -- 172.21.15.138:0/3394757606 >> 172.21.15.138:0/3394757606 conn(0x55983673f800 msgr2=0x5598364a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 -- 172.21.15.138:0/3394757606 shutdown_connections 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 -- 172.21.15.138:0/3394757606 wait complete. 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.572+0000 7f17e7e98880 1 Processor -- start 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e7e98880 1 -- start start 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e7e98880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e7e98880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363ba780 con 0x559836714c00 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e5e4a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e5e4a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43300/0 (socket says 172.21.15.138:43300) 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e5e4a640 1 -- 172.21.15.138:0/526201229 learned_addr learned my addr 172.21.15.138:0/526201229 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e5e4a640 1 -- 172.21.15.138:0/526201229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598364084e0 con 0x559836714c00 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e5e4a640 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x5598366f7860 tx=0x5598366daae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17cb615640 1 -- 172.21.15.138:0/526201229 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559836740840 con 0x559836714c00 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17cb615640 1 -- 172.21.15.138:0/526201229 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559837176a80 con 0x559836714c00 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55983639b380 con 0x559836714c00 2024-04-27T05:21:35.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17cb615640 1 -- 172.21.15.138:0/526201229 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559836740580 con 0x559836714c00 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.573+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55983639aea0 con 0x559836714c00 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.574+0000 7f17cb615640 1 -- 172.21.15.138:0/526201229 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5598363c1b80 con 0x559836714c00 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.574+0000 7f17cb615640 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5598372fac00 0x55983667b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.574+0000 7f17cb615640 1 -- 172.21.15.138:0/526201229 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5598363a9dc0 con 0x559836714c00 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.574+0000 7f17e7e98880 5 note: GC not initialized 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.574+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5598372fac00 0x55983667b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.574+0000 7f17c8e10640 20 reqs_thread_entry: start 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e7e98880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e6e4c640 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5598372fac00 0x55983667b700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5598372f8c30 tx=0x5598366da8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e7e98880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17c6e0c640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:05 2024 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17c65ea640 20 reqs_thread_entry: start 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e7e98880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836810000 con 0x55983718f000 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.575+0000 7f17e7e98880 20 rados_obj.operate() r=0 bl.length=621 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.598 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo2", 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "" 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.599 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo1", 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo2", 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.600 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.601 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.602 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.576+0000 7f17cd619640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.576+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94112236121344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836810400 con 0x55983718f000 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.577+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94112236121344] v29'87 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.577+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94112236112128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836810800 con 0x559836715000 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.578+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94112236112128] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.578+0000 7f17e7e98880 20 remove_watcher() i=0 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.578+0000 7f17e7e98880 2 removed watcher, disabling cache 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.578+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94112236113280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836810c00 con 0x55983718f000 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.579+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94112236113280] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.579+0000 7f17e7e98880 20 remove_watcher() i=1 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.579+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94112236114432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836811000 con 0x559836715800 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.580+0000 7f17e664b640 1 -- 172.21.15.138:0/174706570 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94112236114432] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734fc0 con 0x559836715800 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.580+0000 7f17e7e98880 20 remove_watcher() i=2 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.580+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94112236115584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836811400 con 0x559836715000 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.581+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94112236115584] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836735200 con 0x559836715000 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.581+0000 7f17e7e98880 20 remove_watcher() i=3 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.581+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94112236116736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836811800 con 0x55983718f000 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.582+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94112236116736] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x55983718f000 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.582+0000 7f17e7e98880 20 remove_watcher() i=4 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.582+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94112236117888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836811c00 con 0x559836715c00 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.583+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94112236117888] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.583+0000 7f17e7e98880 20 remove_watcher() i=5 2024-04-27T05:21:35.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.583+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94112236119040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836810000 con 0x559836715c00 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.584+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94112236119040] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.584+0000 7f17e7e98880 20 remove_watcher() i=6 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.584+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94112236120192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x559836810400 con 0x559836715c00 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e6e4c640 1 -- 172.21.15.138:0/174706570 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94112236120192] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559836734900 con 0x559836715c00 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 20 remove_watcher() i=7 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559836715c00 msgr2=0x55983667b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x559836715c00 0x55983667b180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55983718f000 msgr2=0x559836679b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55983718f000 0x559836679b80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559836715000 msgr2=0x55983667ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x559836715000 0x55983667ac00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559836715800 msgr2=0x55983667a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x559836715800 0x55983667a100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559836714000 msgr2=0x559836679600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x559836714000 0x559836679600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5598364a3380 tx=0x5598366da120 comp rx=0 tx=0).stop 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 msgr2=0x559836678b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 --2- 172.21.15.138:0/174706570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55983718ec00 0x559836678b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5598372533b0 tx=0x5598364a9980 comp rx=0 tx=0).stop 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 reap_dead start 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e5e4a640 1 -- 172.21.15.138:0/174706570 reap_dead start 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 shutdown_connections 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 >> 172.21.15.138:0/174706570 conn(0x559836397400 msgr2=0x5598364a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 shutdown_connections 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/174706570 wait complete. 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.585+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5598372fac00 msgr2=0x55983667b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5598372fac00 0x55983667b700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5598372f8c30 tx=0x5598366da8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 msgr2=0x55983667a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x5598366f7860 tx=0x5598366daae0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 shutdown_connections 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5598372fac00 0x55983667b700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/526201229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836714c00 0x55983667a680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 >> 172.21.15.138:0/526201229 conn(0x55983673f800 msgr2=0x5598364a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 shutdown_connections 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/526201229 wait complete. 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55983718fc00 msgr2=0x559836679080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55983718fc00 0x559836679080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x559837298180 tx=0x5598364a8cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 msgr2=0x559836678580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5598372532c0 tx=0x5598364a8c60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 shutdown_connections 2024-04-27T05:21:35.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55983718fc00 0x559836679080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 --2- 172.21.15.138:0/693963953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559836397000 0x559836678580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 >> 172.21.15.138:0/693963953 conn(0x559836396c00 msgr2=0x55983649f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 shutdown_connections 2024-04-27T05:21:35.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.586+0000 7f17e7e98880 1 -- 172.21.15.138:0/693963953 wait complete. 2024-04-27T05:21:35.613 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.613 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-27T05:21:35.613 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-27T05:21:35.613 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-04-27T05:21:35.694 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.694 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.691+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.691+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.691+0000 7fbd5e532880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.691+0000 7fbd5e532880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bef98d8600 con 0x55bef9601400 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.691+0000 7fbd5e532880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bef9640280 con 0x55bef9601000 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.692+0000 7fbd5af11640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.692+0000 7fbd5af11640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43304/0 (socket says 172.21.15.138:43304) 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.692+0000 7fbd5af11640 1 -- 172.21.15.138:0/2298443098 learned_addr learned my addr 172.21.15.138:0/2298443098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.692+0000 7fbd5a710640 1 --1- 172.21.15.138:0/2298443098 >> v1:172.21.15.114:6789/0 conn(0x55bef9601400 0x55bef9708800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60862/0 (socket says 172.21.15.138:60862) 2024-04-27T05:21:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.692+0000 7fbd5af11640 1 -- 172.21.15.138:0/2298443098 >> v1:172.21.15.114:6789/0 conn(0x55bef9601400 legacy=0x55bef9708800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.692+0000 7fbd5af11640 1 -- 172.21.15.138:0/2298443098 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef9605380 con 0x55bef9601000 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2298443098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55bef970cd50 tx=0x55bef9712de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb232d9cf8904a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd59f0f640 1 -- 172.21.15.138:0/2298443098 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619ce0 con 0x55bef9601000 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd59f0f640 1 -- 172.21.15.138:0/2298443098 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bef98d9b00 con 0x55bef9601000 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd59f0f640 1 -- 172.21.15.138:0/2298443098 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619e40 con 0x55bef9601000 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 -- 172.21.15.138:0/2298443098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 msgr2=0x55bef98e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2298443098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55bef970cd50 tx=0x55bef9712de0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 -- 172.21.15.138:0/2298443098 shutdown_connections 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2298443098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 -- 172.21.15.138:0/2298443098 >> 172.21.15.138:0/2298443098 conn(0x55bef9600c00 msgr2=0x55bef9708000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 -- 172.21.15.138:0/2298443098 shutdown_connections 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.694+0000 7fbd5e532880 1 -- 172.21.15.138:0/2298443098 wait complete. 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5e532880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5e532880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9640780 con 0x55bef9601000 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43318/0 (socket says 172.21.15.138:43318) 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.695+0000 7fbd5af11640 1 -- 172.21.15.138:0/2362295770 learned_addr learned my addr 172.21.15.138:0/2362295770 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5af11640 1 -- 172.21.15.138:0/2362295770 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef9604ea0 con 0x55bef9601000 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2362295770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55befa405560 tx=0x55bef9712cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d628fba8e0aad43 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5970e640 1 -- 172.21.15.138:0/2362295770 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef96194a0 con 0x55bef9601000 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5970e640 1 -- 172.21.15.138:0/2362295770 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bef98d9e00 con 0x55bef9601000 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5970e640 1 -- 172.21.15.138:0/2362295770 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619ce0 con 0x55bef9601000 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 -- 172.21.15.138:0/2362295770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 msgr2=0x55bef98e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2362295770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55befa405560 tx=0x55bef9712cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 -- 172.21.15.138:0/2362295770 shutdown_connections 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2362295770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 -- 172.21.15.138:0/2362295770 >> 172.21.15.138:0/2362295770 conn(0x55bef9600c00 msgr2=0x55bef9709000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 -- 172.21.15.138:0/2362295770 shutdown_connections 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.696+0000 7fbd5e532880 1 -- 172.21.15.138:0/2362295770 wait complete. 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5e532880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5e532880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef96408c0 con 0x55bef9601000 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43326/0 (socket says 172.21.15.138:43326) 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5af11640 1 -- 172.21.15.138:0/2408567963 learned_addr learned my addr 172.21.15.138:0/2408567963 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5af11640 1 -- 172.21.15.138:0/2408567963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef96056c0 con 0x55bef9601000 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55befa4bd2c0 tx=0x55bef9712c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.697+0000 7fbd57f0b640 1 -- 172.21.15.138:0/2408567963 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619e40 con 0x55bef9601000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd57f0b640 1 -- 172.21.15.138:0/2408567963 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55befa3e0000 con 0x55bef9601000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bef96051e0 con 0x55bef9601000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd57f0b640 1 -- 172.21.15.138:0/2408567963 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef96194a0 con 0x55bef9601000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bef9672ea0 con 0x55bef9601000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd57f0b640 1 -- 172.21.15.138:0/2408567963 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bef962ab00 con 0x55bef9601000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd57f0b640 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55befa3f8c00 0x55bef98e3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd5e532880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd5e532880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9624280 con 0x55befa3f9000 2024-04-27T05:21:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.698+0000 7fbd57f0b640 1 -- 172.21.15.138:0/2408567963 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55bef9613a40 con 0x55bef9601000 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5b712640 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55befa3f8c00 0x55bef98e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43336/0 (socket says 172.21.15.138:43336) 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5af11640 1 -- 172.21.15.138:0/1690535620 learned_addr learned my addr 172.21.15.138:0/1690535620 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5af11640 1 -- 172.21.15.138:0/1690535620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef9672d00 con 0x55befa3f9000 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5b712640 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55befa3f8c00 0x55bef98e3080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55befa4055c0 tx=0x55bef9713c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5af11640 1 --2- 172.21.15.138:0/1690535620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55befa502240 tx=0x55bef9712cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fdfcfd1fe3f1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd55706640 1 -- 172.21.15.138:0/1690535620 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619ce0 con 0x55befa3f9000 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd55706640 1 -- 172.21.15.138:0/1690535620 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55befa3e0300 con 0x55befa3f9000 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd55706640 1 -- 172.21.15.138:0/1690535620 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619e40 con 0x55befa3f9000 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 -- 172.21.15.138:0/1690535620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 msgr2=0x55bef98e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 --2- 172.21.15.138:0/1690535620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55befa502240 tx=0x55bef9712cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 -- 172.21.15.138:0/1690535620 shutdown_connections 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 --2- 172.21.15.138:0/1690535620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 -- 172.21.15.138:0/1690535620 >> 172.21.15.138:0/1690535620 conn(0x55bef9601400 msgr2=0x55bef970b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 -- 172.21.15.138:0/1690535620 shutdown_connections 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.699+0000 7fbd5e532880 1 -- 172.21.15.138:0/1690535620 wait complete. 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5e532880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5e532880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9625180 con 0x55befa3f9000 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5af11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43348/0 (socket says 172.21.15.138:43348) 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 learned_addr learned my addr 172.21.15.138:0/2037391212 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef9672b60 con 0x55befa3f9000 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.700+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55befa503d70 tx=0x55bef9713aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd53f03640 1 -- 172.21.15.138:0/2037391212 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef96194a0 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd53f03640 1 -- 172.21.15.138:0/2037391212 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55befa3e0480 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bef96729c0 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd53f03640 1 -- 172.21.15.138:0/2037391212 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef9619ce0 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bef9672820 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd53f03640 1 -- 172.21.15.138:0/2037391212 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bef962b600 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd53f03640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef9952000 0x55bef98e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd53f03640 1 -- 172.21.15.138:0/2037391212 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55bef9613c00 con 0x55befa3f9000 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd5a710640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef9952000 0x55bef98e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.701+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55befa3f9400 0x55bef98e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9800 con 0x55befa3f9400 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5b712640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55befa3f9400 0x55bef98e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5a710640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef9952000 0x55bef98e3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55bef970d380 tx=0x55befa57e540 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5b712640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55befa3f9400 0x55bef98e3b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 20 realm 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9c00 con 0x55befa3f9400 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.702+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953000 con 0x55befa3f9400 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953400 con 0x55befa3f9400 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bef9953800 0x55bef98e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953c00 con 0x55bef9953800 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bef9953800 0x55bef98e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bef9953800 0x55bef98e4100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 20 searching for the correct realm 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.703+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953000 con 0x55bef9953800 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5af11640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bef9953000 0x55bef98e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953c00 con 0x55bef9953000 2024-04-27T05:21:35.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5a710640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bef9953000 0x55bef98e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5a710640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bef9953000 0x55bef98e4c00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5a710640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bef9953c00 0x55bef98e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.704+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.705+0000 7fbd5b712640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bef9953c00 0x55bef98e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.705+0000 7fbd5b712640 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bef9953c00 0x55bef98e5180 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.705+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.705+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953000 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.706+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953000 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.707+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.708+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.709+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953000 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.710+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953000 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953800 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953c00 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55befa3f9400 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9952c00 con 0x55bef9953000 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.711+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9800 con 0x55befa3f9400 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9c00 con 0x55befa3f9400 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa560800 con 0x55bef9953800 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 zone default found 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 4 Realm: () 2024-04-27T05:21:35.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 zonegroup default 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.712+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c000 con 0x55bef9953c00 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c400 con 0x55bef9953800 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c800 con 0x55bef9953800 2024-04-27T05:21:35.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 started sync module instance, tier type = 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.713+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997cc00 con 0x55bef9953000 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.714+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.714+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94278718957824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d000 con 0x55bef9953000 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.714+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d400 con 0x55befa3f9400 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.715+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94278718957824] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.715+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.715+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94278718958976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d800 con 0x55befa3f9400 2024-04-27T05:21:35.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.715+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c000 con 0x55bef9953800 2024-04-27T05:21:35.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.716+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94278718958976] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.716+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.716+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94278718960128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d400 con 0x55bef9953800 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.716+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997dc00 con 0x55bef9953000 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.717+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.717+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94278718961280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9800 con 0x55bef9953000 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.717+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94278718960128] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.717+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9c00 con 0x55befa3f9400 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.718+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.718+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94278718962432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953400 con 0x55befa3f9400 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.718+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997cc00 con 0x55bef9953c00 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.718+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94278718961280] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.719+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.719+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94278718963584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d000 con 0x55bef9953c00 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.719+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997dc00 con 0x55bef9953c00 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.719+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94278718962432] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.720+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.720+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94278718964736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c400 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.720+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c800 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.720+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94278718963584] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94278718964736] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94278718965888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c000 con 0x55bef9953c00 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=0 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=1 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=2 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=3 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=4 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=5 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.721+0000 7fbd5e532880 20 add_watcher() i=6 2024-04-27T05:21:35.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.722+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94278718965888] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.722+0000 7fbd5e532880 20 add_watcher() i=7 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.722+0000 7fbd5e532880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.722+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa561800 con 0x55befa3f9400 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.722+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.722+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94278718967040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa560800 con 0x55befa3f9400 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.723+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94278718967040] v29'88 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.723+0000 7fbd41edf640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.723+0000 7fbd5e532880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.723+0000 7fbd5e532880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.723+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.723+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5e532880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5e532880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9624780 con 0x55bef997d800 2024-04-27T05:21:35.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5a710640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5a710640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43350/0 (socket says 172.21.15.138:43350) 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5a710640 1 -- 172.21.15.138:0/1437479613 learned_addr learned my addr 172.21.15.138:0/1437479613 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5a710640 1 -- 172.21.15.138:0/1437479613 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef9672680 con 0x55bef997d800 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd5a710640 1 --2- 172.21.15.138:0/1437479613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55bef99347b0 tx=0x55bef9713bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3df43427df7e28a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd416de640 1 -- 172.21.15.138:0/1437479613 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef997e160 con 0x55bef997d800 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd416de640 1 -- 172.21.15.138:0/1437479613 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55befa3e0c00 con 0x55bef997d800 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.724+0000 7fbd416de640 1 -- 172.21.15.138:0/1437479613 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef997e2c0 con 0x55bef997d800 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 -- 172.21.15.138:0/1437479613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 msgr2=0x55bef98e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 --2- 172.21.15.138:0/1437479613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55bef99347b0 tx=0x55bef9713bc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 -- 172.21.15.138:0/1437479613 shutdown_connections 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 --2- 172.21.15.138:0/1437479613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 -- 172.21.15.138:0/1437479613 >> 172.21.15.138:0/1437479613 conn(0x55bef9952c00 msgr2=0x55bef970b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 -- 172.21.15.138:0/1437479613 shutdown_connections 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 -- 172.21.15.138:0/1437479613 wait complete. 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 Processor -- start 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 -- start start 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.725+0000 7fbd5e532880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5e532880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bef9624500 con 0x55bef997d800 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5a710640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5a710640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43354/0 (socket says 172.21.15.138:43354) 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5a710640 1 -- 172.21.15.138:0/4137453160 learned_addr learned my addr 172.21.15.138:0/4137453160 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5a710640 1 -- 172.21.15.138:0/4137453160 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bef96724e0 con 0x55bef997d800 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5a710640 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55bef99357a0 tx=0x55befa57ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd3fedb640 1 -- 172.21.15.138:0/4137453160 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef997e420 con 0x55bef997d800 2024-04-27T05:21:35.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd3fedb640 1 -- 172.21.15.138:0/4137453160 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55befa3e0d80 con 0x55bef997d800 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd3fedb640 1 -- 172.21.15.138:0/4137453160 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bef997e160 con 0x55bef997d800 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bef9605380 con 0x55bef997d800 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.726+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bef9604ea0 con 0x55bef997d800 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.727+0000 7fbd3fedb640 1 -- 172.21.15.138:0/4137453160 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bef962bb80 con 0x55bef997d800 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.727+0000 7fbd3fedb640 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef998b000 0x55bef98e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.727+0000 7fbd3fedb640 1 -- 172.21.15.138:0/4137453160 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55bef99e0000 con 0x55bef997d800 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.727+0000 7fbd5b712640 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef998b000 0x55bef98e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.727+0000 7fbd5e532880 5 note: GC not initialized 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd3d6d6640 20 reqs_thread_entry: start 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5b712640 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef998b000 0x55bef98e5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55bef9934e70 tx=0x55befa57eea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd3b6d2640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:35.751 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd3aeb0640 20 reqs_thread_entry: start 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d000 con 0x55befa3f9400 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 20 rados_obj.operate() r=0 bl.length=621 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.728+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~539 in=539b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d400 con 0x55befa3f9400 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.729+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~539] v29'9 uv9 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.729+0000 7fbd5e532880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.729+0000 7fbd5e532880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.729+0000 7fbd5e532880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.729+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94278718969344 in=727b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953400 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94278718962432 notify 124554051592 ret 0) v3 ==== 757+0+0 (crc 0 0 0) 0x55bef9a56000 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd55706640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051592 cookie 94278718962432 notifier 4429 bl.length()=715 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd55706640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd55706640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 94278718969344 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd55706640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c400 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94278718969344 notify 124554051592 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bef9a561a0 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.730+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55befa3f9c00 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(66 foo:foo1 [delete] v29'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd5e532880 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-27T05:21:35.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94278718969344 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef998b400 con 0x55bef9953000 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== watch-notify(notify (1) cookie 94278718957824 notify 124554051588 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x55bef99d7d40 con 0x55bef9953000 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd54f05640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051588 cookie 94278718957824 notifier 4429 bl.length()=159 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd54f05640 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9a5e000 con 0x55bef9953000 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.731+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.0 [notify cookie 94278718969344 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.732+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.732+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== watch-notify(notify_complete (2) cookie 94278718969344 notify 124554051588 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bef99d7a00 con 0x55bef9953000 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.754 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo1", 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "full-control" 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo2", 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "" 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.755 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo2", 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.756 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.757 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.758 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.732+0000 7fbd41edf640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.732+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94278718967040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef998b800 con 0x55befa3f9400 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.733+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94278718967040] v29'89 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.733+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94278718957824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef998bc00 con 0x55bef9953000 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.734+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94278718957824] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.734+0000 7fbd5e532880 20 remove_watcher() i=0 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.734+0000 7fbd5e532880 2 removed watcher, disabling cache 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.734+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94278718958976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef9953400 con 0x55befa3f9400 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.735+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94278718958976] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.735+0000 7fbd5e532880 20 remove_watcher() i=1 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.735+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94278718960128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997c000 con 0x55bef9953800 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.736+0000 7fbd5af11640 1 -- 172.21.15.138:0/2037391212 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94278718960128] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996ed80 con 0x55bef9953800 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.736+0000 7fbd5e532880 20 remove_watcher() i=2 2024-04-27T05:21:35.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.736+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94278718961280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d000 con 0x55bef9953000 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.737+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94278718961280] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996efc0 con 0x55bef9953000 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.737+0000 7fbd5e532880 20 remove_watcher() i=3 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.737+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94278718962432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d400 con 0x55befa3f9400 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.738+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94278718962432] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55befa3f9400 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.738+0000 7fbd5e532880 20 remove_watcher() i=4 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.738+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94278718963584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997dc00 con 0x55bef9953c00 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.739+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94278718963584] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.739+0000 7fbd5e532880 20 remove_watcher() i=5 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.739+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94278718964736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef997d000 con 0x55bef9953c00 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.740+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94278718964736] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.740+0000 7fbd5e532880 20 remove_watcher() i=6 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.740+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94278718965888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55bef998ac00 con 0x55bef9953c00 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5b712640 1 -- 172.21.15.138:0/2037391212 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94278718965888] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bef996e6c0 con 0x55bef9953c00 2024-04-27T05:21:35.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 20 remove_watcher() i=7 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bef9953c00 msgr2=0x55bef98e5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bef9953c00 0x55bef98e5180 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55befa3f9400 msgr2=0x55bef98e3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55befa3f9400 0x55bef98e3b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bef9953000 msgr2=0x55bef98e4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bef9953000 0x55bef98e4c00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bef9953800 msgr2=0x55bef98e4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bef9953800 0x55bef98e4100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef9952000 msgr2=0x55bef98e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef9952000 0x55bef98e3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55bef970d380 tx=0x55befa57e540 comp rx=0 tx=0).stop 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 msgr2=0x55bef98e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2037391212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55befa3f9000 0x55bef98e2b00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55befa503d70 tx=0x55bef9713aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 reap_dead start 2024-04-27T05:21:35.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 shutdown_connections 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5a710640 1 -- 172.21.15.138:0/2037391212 reap_dead start 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 >> 172.21.15.138:0/2037391212 conn(0x55bef9601400 msgr2=0x55bef970b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 shutdown_connections 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/2037391212 wait complete. 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef998b000 msgr2=0x55bef98e5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef998b000 0x55bef98e5700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55bef9934e70 tx=0x55befa57eea0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 msgr2=0x55bef98e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55bef99357a0 tx=0x55befa57ec00 comp rx=0 tx=0).stop 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 shutdown_connections 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bef998b000 0x55bef98e5700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 --2- 172.21.15.138:0/4137453160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef997d800 0x55bef98e4680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 >> 172.21.15.138:0/4137453160 conn(0x55bef9952c00 msgr2=0x55bef970b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 shutdown_connections 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.741+0000 7fbd5e532880 1 -- 172.21.15.138:0/4137453160 wait complete. 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55befa3f8c00 msgr2=0x55bef98e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55befa3f8c00 0x55bef98e3080 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55befa4055c0 tx=0x55bef9713c20 comp rx=0 tx=0).stop 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 msgr2=0x55bef98e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55befa4bd2c0 tx=0x55bef9712c60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 shutdown_connections 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55befa3f8c00 0x55bef98e3080 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 --2- 172.21.15.138:0/2408567963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bef9601000 0x55bef98e2580 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 >> 172.21.15.138:0/2408567963 conn(0x55bef9600c00 msgr2=0x55bef9709000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 shutdown_connections 2024-04-27T05:21:35.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.742+0000 7fbd5e532880 1 -- 172.21.15.138:0/2408567963 wait complete. 2024-04-27T05:21:35.768 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.769 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-27T05:21:35.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-27T05:21:35.769 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-04-27T05:21:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.812 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f878bf27880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56174c6c1400 0x56174c9a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f878bf27880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56174c998600 con 0x56174c6c1000 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f878bf27880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x56174c700280 con 0x56174c6c1400 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f87880cd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56174c6c1400 0x56174c9a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f87880cd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56174c6c1400 0x56174c9a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43362/0 (socket says 172.21.15.138:43362) 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f87888ce640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x56174c6c1000 0x56174c7c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60874/0 (socket says 172.21.15.138:60874) 2024-04-27T05:21:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f87880cd640 1 -- 172.21.15.138:0/897309159 learned_addr learned my addr 172.21.15.138:0/897309159 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f87880cd640 1 -- 172.21.15.138:0/897309159 >> v1:172.21.15.114:6789/0 conn(0x56174c6c1000 legacy=0x56174c7c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.810+0000 7f87880cd640 1 -- 172.21.15.138:0/897309159 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c6c5380 con 0x56174c6c1400 2024-04-27T05:21:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.812+0000 7f87880cd640 1 --2- 172.21.15.138:0/897309159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1400 0x56174c9a2580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x56174c7ccc00 tx=0x56174c7d2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ba12447323a579 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.812+0000 7f87878cc640 1 -- 172.21.15.138:0/897309159 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d9ce0 con 0x56174c6c1400 2024-04-27T05:21:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.812+0000 7f87878cc640 1 -- 172.21.15.138:0/897309159 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174c999b00 con 0x56174c6c1400 2024-04-27T05:21:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.812+0000 7f87878cc640 1 -- 172.21.15.138:0/897309159 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d9e40 con 0x56174c6c1400 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- 172.21.15.138:0/897309159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1400 msgr2=0x56174c9a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 --2- 172.21.15.138:0/897309159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1400 0x56174c9a2580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x56174c7ccc00 tx=0x56174c7d2de0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- 172.21.15.138:0/897309159 shutdown_connections 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 --2- 172.21.15.138:0/897309159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1400 0x56174c9a2580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- 172.21.15.138:0/897309159 >> 172.21.15.138:0/897309159 conn(0x56174c6c0c00 msgr2=0x56174c7c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- 172.21.15.138:0/897309159 shutdown_connections 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- 172.21.15.138:0/897309159 wait complete. 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.813+0000 7f878bf27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c700780 con 0x56174c6c1000 2024-04-27T05:21:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87888ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87888ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43364/0 (socket says 172.21.15.138:43364) 2024-04-27T05:21:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87888ce640 1 -- 172.21.15.138:0/894905165 learned_addr learned my addr 172.21.15.138:0/894905165 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87888ce640 1 -- 172.21.15.138:0/894905165 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c6c4ea0 con 0x56174c6c1000 2024-04-27T05:21:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87888ce640 1 --2- 172.21.15.138:0/894905165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56174d57d290 tx=0x56174c7d2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76a80fc04cd062c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87870cb640 1 -- 172.21.15.138:0/894905165 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d94a0 con 0x56174c6c1000 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87870cb640 1 -- 172.21.15.138:0/894905165 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174c999e00 con 0x56174c6c1000 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.814+0000 7f87870cb640 1 -- 172.21.15.138:0/894905165 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d9ce0 con 0x56174c6c1000 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- 172.21.15.138:0/894905165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 msgr2=0x56174c9a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 --2- 172.21.15.138:0/894905165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56174d57d290 tx=0x56174c7d2cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- 172.21.15.138:0/894905165 shutdown_connections 2024-04-27T05:21:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 --2- 172.21.15.138:0/894905165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- 172.21.15.138:0/894905165 >> 172.21.15.138:0/894905165 conn(0x56174c6c0c00 msgr2=0x56174c7c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- 172.21.15.138:0/894905165 shutdown_connections 2024-04-27T05:21:35.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- 172.21.15.138:0/894905165 wait complete. 2024-04-27T05:21:35.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f878bf27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c7008c0 con 0x56174c6c1000 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f87888ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f87888ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43380/0 (socket says 172.21.15.138:43380) 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f87888ce640 1 -- 172.21.15.138:0/2672730495 learned_addr learned my addr 172.21.15.138:0/2672730495 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.815+0000 7f87888ce640 1 -- 172.21.15.138:0/2672730495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c6c56c0 con 0x56174c6c1000 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f87888ce640 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x56174d57cb40 tx=0x56174c7d2c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f87858c8640 1 -- 172.21.15.138:0/2672730495 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d9e40 con 0x56174c6c1000 2024-04-27T05:21:35.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f87858c8640 1 -- 172.21.15.138:0/2672730495 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174d592000 con 0x56174c6c1000 2024-04-27T05:21:35.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56174c6c51e0 con 0x56174c6c1000 2024-04-27T05:21:35.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f87858c8640 1 -- 172.21.15.138:0/2672730495 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d94a0 con 0x56174c6c1000 2024-04-27T05:21:35.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56174c732ea0 con 0x56174c6c1000 2024-04-27T05:21:35.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f87858c8640 1 -- 172.21.15.138:0/2672730495 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56174c6eb600 con 0x56174c6c1000 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f878bf27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.816+0000 7f878bf27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c6e4280 con 0x56174d4b9800 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87858c8640 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174d616000 0x56174c9a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87890cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87890cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43390/0 (socket says 172.21.15.138:43390) 2024-04-27T05:21:35.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87890cf640 1 -- 172.21.15.138:0/2655344502 learned_addr learned my addr 172.21.15.138:0/2655344502 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87858c8640 1 -- 172.21.15.138:0/2672730495 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56174c6d3a40 con 0x56174c6c1000 2024-04-27T05:21:35.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87888ce640 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174d616000 0x56174c9a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87890cf640 1 -- 172.21.15.138:0/2655344502 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c732d00 con 0x56174d4b9800 2024-04-27T05:21:35.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87890cf640 1 --2- 172.21.15.138:0/2655344502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56174d57dd10 tx=0x56174c7d2f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c1fce50e7da7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87888ce640 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174d616000 0x56174c9a3080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x56174d5c6a20 tx=0x56174c7d34a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87830c3640 1 -- 172.21.15.138:0/2655344502 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d9e40 con 0x56174d4b9800 2024-04-27T05:21:35.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87830c3640 1 -- 172.21.15.138:0/2655344502 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174d592300 con 0x56174d4b9800 2024-04-27T05:21:35.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f87830c3640 1 -- 172.21.15.138:0/2655344502 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d94a0 con 0x56174d4b9800 2024-04-27T05:21:35.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f878bf27880 1 -- 172.21.15.138:0/2655344502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 msgr2=0x56174c9a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f878bf27880 1 --2- 172.21.15.138:0/2655344502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56174d57dd10 tx=0x56174c7d2f00 comp rx=0 tx=0).stop 2024-04-27T05:21:35.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f878bf27880 1 -- 172.21.15.138:0/2655344502 shutdown_connections 2024-04-27T05:21:35.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f878bf27880 1 --2- 172.21.15.138:0/2655344502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.817+0000 7f878bf27880 1 -- 172.21.15.138:0/2655344502 >> 172.21.15.138:0/2655344502 conn(0x56174c6c1400 msgr2=0x56174c7cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f878bf27880 1 -- 172.21.15.138:0/2655344502 shutdown_connections 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f878bf27880 1 -- 172.21.15.138:0/2655344502 wait complete. 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f878bf27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f878bf27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c6e5180 con 0x56174d4b9800 2024-04-27T05:21:35.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f87890cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f87890cf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43402/0 (socket says 172.21.15.138:43402) 2024-04-27T05:21:35.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 learned_addr learned my addr 172.21.15.138:0/846206357 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c732b60 con 0x56174d4b9800 2024-04-27T05:21:35.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.818+0000 7f87890cf640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x56174d57d9e0 tx=0x56174c7d3aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87818c0640 1 -- 172.21.15.138:0/846206357 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d9e40 con 0x56174d4b9800 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87818c0640 1 -- 172.21.15.138:0/846206357 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174d592480 con 0x56174d4b9800 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56174c7329c0 con 0x56174d4b9800 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87818c0640 1 -- 172.21.15.138:0/846206357 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174c6d94a0 con 0x56174d4b9800 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56174c732820 con 0x56174d4b9800 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87818c0640 1 -- 172.21.15.138:0/846206357 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56174c6eb8c0 con 0x56174d4b9800 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87818c0640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca44000 0x56174c9a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87818c0640 1 -- 172.21.15.138:0/846206357 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56174c6d3c00 con 0x56174d4b9800 2024-04-27T05:21:35.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.819+0000 7f87880cd640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca44000 0x56174c9a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56174d4b9c00 0x56174c9a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44400 con 0x56174d4b9c00 2024-04-27T05:21:35.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f87890cf640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56174d4b9c00 0x56174c9a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f87880cd640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca44000 0x56174c9a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x56174c7ccc00 tx=0x56174ca0a120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f87890cf640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56174d4b9c00 0x56174c9a3b80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 20 realm 2024-04-27T05:21:35.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44800 con 0x56174d4b9c00 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.820+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44c00 con 0x56174d4b9c00 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45000 con 0x56174d4b9c00 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56174ca45400 0x56174c9a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44400 con 0x56174ca45400 2024-04-27T05:21:35.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f87888ce640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56174ca45400 0x56174c9a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.821+0000 7f87888ce640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56174ca45400 0x56174c9a4100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f878bf27880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:35.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f878bf27880 20 searching for the correct realm 2024-04-27T05:21:35.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44800 con 0x56174ca45400 2024-04-27T05:21:35.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f87888ce640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56174ca44800 0x56174c9a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44400 con 0x56174ca44800 2024-04-27T05:21:35.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f87880cd640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56174ca44800 0x56174c9a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.822+0000 7f87880cd640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56174ca44800 0x56174c9a4c00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87880cd640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56174ca44400 0x56174c9a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87890cf640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56174ca44400 0x56174c9a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87890cf640 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56174ca44400 0x56174c9a5180 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.823+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44800 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.824+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.825+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44800 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.826+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.827+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44800 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.828+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44800 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca45400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.829+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174d4b9c00 2024-04-27T05:21:35.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d4b8800 con 0x56174ca44800 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44c00 con 0x56174d4b9c00 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45000 con 0x56174d4b9c00 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.830+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45800 con 0x56174ca45400 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 zone default found 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 4 Realm: () 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 zonegroup default 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45c00 con 0x56174ca44400 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56000 con 0x56174ca45400 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.831+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56400 con 0x56174ca45400 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.832+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.832+0000 7f878bf27880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.832+0000 7f878bf27880 20 started sync module instance, tier type = 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.832+0000 7f878bf27880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.832+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56800 con 0x56174ca44800 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.833+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.833+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94658069375232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56c00 con 0x56174ca44800 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.833+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca57000 con 0x56174d4b9c00 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.834+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94658069375232] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.834+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.834+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94658069376384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca57400 con 0x56174d4b9c00 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.834+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca57800 con 0x56174ca45400 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.834+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94658069376384] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.835+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.835+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94658069377536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56000 con 0x56174ca45400 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.835+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56400 con 0x56174ca44800 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.837+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94658069377536] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.837+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.837+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94658069378688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca57800 con 0x56174ca44800 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.837+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45800 con 0x56174d4b9c00 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.838+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94658069378688] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.838+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.838+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94658069379840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca44c00 con 0x56174d4b9c00 2024-04-27T05:21:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.838+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45000 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.839+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.839+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94658069379840] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.839+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94658069380992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca45c00 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.839+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174d617800 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.840+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94658069380992] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.840+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.840+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94658069382144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56800 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.840+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56c00 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94658069383296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca56400 con 0x56174ca44400 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=0 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=1 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=2 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=3 2024-04-27T05:21:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=4 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=5 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94658069382144] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.841+0000 7f878bf27880 20 add_watcher() i=6 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.842+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94658069383296] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.842+0000 7f878bf27880 20 add_watcher() i=7 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.842+0000 7f878bf27880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.842+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca57800 con 0x56174d4b9c00 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.842+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.842+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94658069384448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174ca57000 con 0x56174d4b9c00 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.843+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94658069384448] v29'90 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.843+0000 7f876f89c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.843+0000 7f878bf27880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.843+0000 7f878bf27880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.843+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.843+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f878bf27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f878bf27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c6e4780 con 0x56174ca57c00 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f87880cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f87880cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43412/0 (socket says 172.21.15.138:43412) 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f87880cd640 1 -- 172.21.15.138:0/3249700393 learned_addr learned my addr 172.21.15.138:0/3249700393 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f87880cd640 1 -- 172.21.15.138:0/3249700393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c732680 con 0x56174ca57c00 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f87880cd640 1 --2- 172.21.15.138:0/3249700393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x56174d6367b0 tx=0x56174ca0a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7049db61c09e4bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f876f09b640 1 -- 172.21.15.138:0/3249700393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174d4a4580 con 0x56174ca57c00 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f876f09b640 1 -- 172.21.15.138:0/3249700393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174d592000 con 0x56174ca57c00 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.844+0000 7f876f09b640 1 -- 172.21.15.138:0/3249700393 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174d4a46e0 con 0x56174ca57c00 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 -- 172.21.15.138:0/3249700393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 msgr2=0x56174c9a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 --2- 172.21.15.138:0/3249700393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x56174d6367b0 tx=0x56174ca0a0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 -- 172.21.15.138:0/3249700393 shutdown_connections 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 --2- 172.21.15.138:0/3249700393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 -- 172.21.15.138:0/3249700393 >> 172.21.15.138:0/3249700393 conn(0x56174ca57400 msgr2=0x56174c7cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 -- 172.21.15.138:0/3249700393 shutdown_connections 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 -- 172.21.15.138:0/3249700393 wait complete. 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 Processor -- start 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.845+0000 7f878bf27880 1 -- start start 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f878bf27880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f878bf27880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56174c6e4500 con 0x56174ca57c00 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f87880cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f87880cd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43418/0 (socket says 172.21.15.138:43418) 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f87880cd640 1 -- 172.21.15.138:0/2053980431 learned_addr learned my addr 172.21.15.138:0/2053980431 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f87880cd640 1 -- 172.21.15.138:0/2053980431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56174c7324e0 con 0x56174ca57c00 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f87880cd640 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x56174ca27620 tx=0x56174ca0a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f876d898640 1 -- 172.21.15.138:0/2053980431 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174d4a4840 con 0x56174ca57c00 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f876d898640 1 -- 172.21.15.138:0/2053980431 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56174d592780 con 0x56174ca57c00 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56174c6c4ea0 con 0x56174ca57c00 2024-04-27T05:21:35.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f876d898640 1 -- 172.21.15.138:0/2053980431 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56174d4a4580 con 0x56174ca57c00 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.846+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56174c6c56c0 con 0x56174ca57c00 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f876d898640 1 -- 172.21.15.138:0/2053980431 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56174c6eab00 con 0x56174ca57c00 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f876d898640 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca56800 0x56174c9a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f876d898640 1 -- 172.21.15.138:0/2053980431 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56174c6d3dc0 con 0x56174ca57c00 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f87890cf640 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca56800 0x56174c9a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f878bf27880 5 note: GC not initialized 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f876b093640 20 reqs_thread_entry: start 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f878bf27880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.847+0000 7f87890cf640 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca56800 0x56174c9a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x56174ca26db0 tx=0x56174ca0a960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f876908f640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f876886d640 20 reqs_thread_entry: start 2024-04-27T05:21:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 20 rados->read ofs=0 len=0 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb36000 con 0x56174d4b9c00 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~539 out=539b,getxattrs out=75b] v0'0 uv9 ondisk = 0) v8 ==== 231+0+662 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 20 rados_obj.operate() r=0 bl.length=539 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.848+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb36800 con 0x56174d4b9c00 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.849+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v29'10 uv10 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.849+0000 7f878bf27880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.849+0000 7f878bf27880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.849+0000 7f878bf27880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.849+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94658069386752 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb36c00 con 0x56174d4b9c00 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94658069379840 notify 124554051593 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x56174ca09d40 con 0x56174d4b9c00 2024-04-27T05:21:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 94658069386752 out=8b] v0'0 uv75 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87830c3640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051593 cookie 94658069379840 notifier 4437 bl.length()=689 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87830c3640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87830c3640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87830c3640 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb42000 con 0x56174d4b9c00 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.850+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94658069386752 notify 124554051593 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56174cabd860 con 0x56174d4b9c00 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [ 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "foo:foo2", 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "permissions": "" 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [ 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo:foo2", 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:35.894 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:35.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.851+0000 7f876f89c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:35.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.851+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94658069384448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb37000 con 0x56174d4b9c00 2024-04-27T05:21:35.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.852+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94658069384448] v29'91 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.852+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94658069375232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb37400 con 0x56174ca44800 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.853+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94658069375232] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.853+0000 7f878bf27880 20 remove_watcher() i=0 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.853+0000 7f878bf27880 2 removed watcher, disabling cache 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.853+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94658069376384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb37800 con 0x56174d4b9c00 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.854+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94658069376384] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.854+0000 7f878bf27880 20 remove_watcher() i=1 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.854+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94658069377536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb37c00 con 0x56174ca45400 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.855+0000 7f87888ce640 1 -- 172.21.15.138:0/846206357 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94658069377536] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54900 con 0x56174ca45400 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.855+0000 7f878bf27880 20 remove_watcher() i=2 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.855+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94658069378688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb42000 con 0x56174ca44800 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.856+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94658069378688] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54b40 con 0x56174ca44800 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.856+0000 7f878bf27880 20 remove_watcher() i=3 2024-04-27T05:21:35.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.856+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94658069379840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb42800 con 0x56174d4b9c00 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.857+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94658069379840] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174d4b9c00 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.857+0000 7f878bf27880 20 remove_watcher() i=4 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.857+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94658069380992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb42c00 con 0x56174ca44400 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.858+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94658069380992] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.858+0000 7f878bf27880 20 remove_watcher() i=5 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.858+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94658069382144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb43000 con 0x56174ca44400 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.859+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94658069382144] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.859+0000 7f878bf27880 20 remove_watcher() i=6 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.859+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94658069383296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56174cb43400 con 0x56174ca44400 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f87890cf640 1 -- 172.21.15.138:0/846206357 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94658069383296] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56174ca54240 con 0x56174ca44400 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 20 remove_watcher() i=7 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56174ca44400 msgr2=0x56174c9a5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56174ca44400 0x56174c9a5180 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56174d4b9c00 msgr2=0x56174c9a3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56174d4b9c00 0x56174c9a3b80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56174ca44800 msgr2=0x56174c9a4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56174ca44800 0x56174c9a4c00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56174ca45400 msgr2=0x56174c9a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56174ca45400 0x56174c9a4100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca44000 msgr2=0x56174c9a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca44000 0x56174c9a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x56174c7ccc00 tx=0x56174ca0a120 comp rx=0 tx=0).stop 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 msgr2=0x56174c9a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/846206357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174d4b9800 0x56174c9a2b00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x56174d57d9e0 tx=0x56174c7d3aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 reap_dead start 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 shutdown_connections 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 >> 172.21.15.138:0/846206357 conn(0x56174c6c1400 msgr2=0x56174c7cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f87880cd640 1 -- 172.21.15.138:0/846206357 reap_dead start 2024-04-27T05:21:35.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 shutdown_connections 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/846206357 wait complete. 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca56800 msgr2=0x56174c9a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca56800 0x56174c9a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x56174ca26db0 tx=0x56174ca0a960 comp rx=0 tx=0).stop 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 msgr2=0x56174c9a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x56174ca27620 tx=0x56174ca0a000 comp rx=0 tx=0).stop 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 shutdown_connections 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174ca56800 0x56174c9a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 --2- 172.21.15.138:0/2053980431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174ca57c00 0x56174c9a4680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.860+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 >> 172.21.15.138:0/2053980431 conn(0x56174ca57400 msgr2=0x56174c7cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 shutdown_connections 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2053980431 wait complete. 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174d616000 msgr2=0x56174c9a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174d616000 0x56174c9a3080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x56174d5c6a20 tx=0x56174c7d34a0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 msgr2=0x56174c9a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x56174d57cb40 tx=0x56174c7d2c60 comp rx=0 tx=0).stop 2024-04-27T05:21:35.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 shutdown_connections 2024-04-27T05:21:35.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56174d616000 0x56174c9a3080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 --2- 172.21.15.138:0/2672730495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56174c6c1000 0x56174c9a2580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 >> 172.21.15.138:0/2672730495 conn(0x56174c6c0c00 msgr2=0x56174c7c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 shutdown_connections 2024-04-27T05:21:35.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.861+0000 7f878bf27880 1 -- 172.21.15.138:0/2672730495 wait complete. 2024-04-27T05:21:35.904 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:35.904 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-27T05:21:35.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-27T05:21:35.905 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-04-27T05:21:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bfcb20880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c236e95000 0x55c237176580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bfcb20880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c23716c600 con 0x55c236e95400 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bfcb20880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c236ed4280 con 0x55c236e95000 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bf9523640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c236e95000 0x55c237176580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bf8d22640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55c236e95400 0x55c236f9c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60882/0 (socket says 172.21.15.138:60882) 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bf9523640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c236e95000 0x55c237176580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43424/0 (socket says 172.21.15.138:43424) 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2238172714 learned_addr learned my addr 172.21.15.138:0/2238172714 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bf9523640 1 -- 172.21.15.138:0/2238172714 >> v1:172.21.15.114:6789/0 conn(0x55c236e95400 legacy=0x55c236f9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.984+0000 7f4bf9523640 1 -- 172.21.15.138:0/2238172714 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236e99380 con 0x55c236e95000 2024-04-27T05:21:35.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.986+0000 7f4bf9523640 1 --2- 172.21.15.138:0/2238172714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55c236fa0d50 tx=0x55c236fa6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edb74f6f84af82c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:35.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.986+0000 7f4bf8521640 1 -- 172.21.15.138:0/2238172714 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236eadce0 con 0x55c236e95000 2024-04-27T05:21:35.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.986+0000 7f4bf8521640 1 -- 172.21.15.138:0/2238172714 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c23716db00 con 0x55c236e95000 2024-04-27T05:21:35.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.986+0000 7f4bf8521640 1 -- 172.21.15.138:0/2238172714 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236eade40 con 0x55c236e95000 2024-04-27T05:21:35.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2238172714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 msgr2=0x55c237176580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:35.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2238172714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55c236fa0d50 tx=0x55c236fa6de0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2238172714 shutdown_connections 2024-04-27T05:21:35.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2238172714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:35.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2238172714 >> 172.21.15.138:0/2238172714 conn(0x55c236e94c00 msgr2=0x55c236f9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:35.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2238172714 shutdown_connections 2024-04-27T05:21:35.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2238172714 wait complete. 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.987+0000 7f4bfcb20880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236ed4780 con 0x55c236e95000 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf9523640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf9523640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43428/0 (socket says 172.21.15.138:43428) 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf9523640 1 -- 172.21.15.138:0/287867464 learned_addr learned my addr 172.21.15.138:0/287867464 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf9523640 1 -- 172.21.15.138:0/287867464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236e98ea0 con 0x55c236e95000 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf9523640 1 --2- 172.21.15.138:0/287867464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55c237c99560 tx=0x55c236fa6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15eb228ba8d3dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf7d20640 1 -- 172.21.15.138:0/287867464 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236ead4a0 con 0x55c236e95000 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf7d20640 1 -- 172.21.15.138:0/287867464 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c23716de00 con 0x55c236e95000 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bf7d20640 1 -- 172.21.15.138:0/287867464 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236eadce0 con 0x55c236e95000 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bfcb20880 1 -- 172.21.15.138:0/287867464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 msgr2=0x55c237176580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.988+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/287867464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55c237c99560 tx=0x55c236fa6cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 -- 172.21.15.138:0/287867464 shutdown_connections 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/287867464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 -- 172.21.15.138:0/287867464 >> 172.21.15.138:0/287867464 conn(0x55c236e94c00 msgr2=0x55c236f9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 -- 172.21.15.138:0/287867464 shutdown_connections 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 -- 172.21.15.138:0/287867464 wait complete. 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bfcb20880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236ed48c0 con 0x55c236e95000 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bf9523640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bf9523640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43434/0 (socket says 172.21.15.138:43434) 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.989+0000 7f4bf9523640 1 -- 172.21.15.138:0/7215209 learned_addr learned my addr 172.21.15.138:0/7215209 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bf9523640 1 -- 172.21.15.138:0/7215209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236e996c0 con 0x55c236e95000 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bf9523640 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55c237d50de0 tx=0x55c236fa6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bf651d640 1 -- 172.21.15.138:0/7215209 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236eade40 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bf651d640 1 -- 172.21.15.138:0/7215209 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c237c74000 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c236e991e0 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bf651d640 1 -- 172.21.15.138:0/7215209 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236ead4a0 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c236f06ea0 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.990+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bfcb20880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bfcb20880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236eb8280 con 0x55c237c8cc00 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf651d640 1 -- 172.21.15.138:0/7215209 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c236ebeb00 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9d24640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf651d640 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237c8dc00 0x55c237177080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9d24640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43446/0 (socket says 172.21.15.138:43446) 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf651d640 1 -- 172.21.15.138:0/7215209 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c236ea7a40 con 0x55c236e95000 2024-04-27T05:21:36.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9d24640 1 -- 172.21.15.138:0/1643031755 learned_addr learned my addr 172.21.15.138:0/1643031755 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9523640 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237c8dc00 0x55c237177080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9d24640 1 -- 172.21.15.138:0/1643031755 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236f06d00 con 0x55c237c8cc00 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/1643031755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55c237d51d10 tx=0x55c236fa7c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2216c1f3833bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf9523640 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237c8dc00 0x55c237177080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c237d942a0 tx=0x55c236fa6ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf3d18640 1 -- 172.21.15.138:0/1643031755 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236eade40 con 0x55c237c8cc00 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf3d18640 1 -- 172.21.15.138:0/1643031755 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c237c74600 con 0x55c237c8cc00 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.991+0000 7f4bf3d18640 1 -- 172.21.15.138:0/1643031755 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236ead4a0 con 0x55c237c8cc00 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- 172.21.15.138:0/1643031755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 msgr2=0x55c237176b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/1643031755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55c237d51d10 tx=0x55c236fa7c20 comp rx=0 tx=0).stop 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- 172.21.15.138:0/1643031755 shutdown_connections 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/1643031755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- 172.21.15.138:0/1643031755 >> 172.21.15.138:0/1643031755 conn(0x55c236e95400 msgr2=0x55c236f9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- 172.21.15.138:0/1643031755 shutdown_connections 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- 172.21.15.138:0/1643031755 wait complete. 2024-04-27T05:21:36.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bfcb20880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236eb9180 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bf9d24640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bf9d24640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43454/0 (socket says 172.21.15.138:43454) 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.992+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 learned_addr learned my addr 172.21.15.138:0/2203927216 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236f06b60 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55c237c995c0 tx=0x55c236fa79e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bf2515640 1 -- 172.21.15.138:0/2203927216 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236eade40 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bf2515640 1 -- 172.21.15.138:0/2203927216 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c237c74780 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c236f069c0 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bf2515640 1 -- 172.21.15.138:0/2203927216 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c236ead4a0 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.993+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c236f06820 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf2515640 1 -- 172.21.15.138:0/2203927216 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c236ebf600 con 0x55c237c8cc00 2024-04-27T05:21:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf2515640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237212000 0x55c237177600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf2515640 1 -- 172.21.15.138:0/2203927216 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c236ea7c00 con 0x55c237c8cc00 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237212000 0x55c237177600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c237c8d000 0x55c237177b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237c8d400 con 0x55c237c8d000 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c237c8d000 0x55c237177b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237212000 0x55c237177600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55c237e01260 tx=0x55c2371d6120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.994+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c237c8d000 0x55c237177b80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 realm 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237c8d800 con 0x55c237c8d000 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213000 con 0x55c237c8d000 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213400 con 0x55c237c8d000 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c237213800 0x55c237178100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213c00 con 0x55c237213800 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.995+0000 7f4bf9523640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c237213800 0x55c237178100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bf9523640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c237213800 0x55c237178100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bfcb20880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bfcb20880 20 searching for the correct realm 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213000 con 0x55c237213800 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bf9523640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c237213000 0x55c237178c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.996+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213c00 con 0x55c237213000 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.997+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c237213000 0x55c237178c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.997+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c237213000 0x55c237178c00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.997+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.997+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c237213c00 0x55c237179180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.997+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.997+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c237213c00 0x55c237179180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c237213c00 0x55c237179180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.998+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:35.999+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213000 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.000+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.001+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213000 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.002+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213000 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213800 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.003+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213c00 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237c8d000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237212c00 con 0x55c237213000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bfcb20880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bfcb20880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bfcb20880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bfcb20880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213400 con 0x55c237c8d000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.004+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237c8d400 con 0x55c237c8d000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237c8d800 con 0x55c237213800 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 zone default found 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 4 Realm: () 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 zonegroup default 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236000 con 0x55c237213c00 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236400 con 0x55c237213800 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.005+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236800 con 0x55c237213800 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.006+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.006+0000 7f4bfcb20880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.006+0000 7f4bfcb20880 20 started sync module instance, tier type = 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.006+0000 7f4bfcb20880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.006+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236c00 con 0x55c237213000 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.007+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.007+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94292636264704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237000 con 0x55c237213000 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.007+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237400 con 0x55c237c8d000 2024-04-27T05:21:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.008+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94292636264704] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.008+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.008+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94292636265856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237800 con 0x55c237c8d000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.008+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237c00 con 0x55c237213800 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.009+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94292636265856] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.009+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.009+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94292636267008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236400 con 0x55c237213800 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.009+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236800 con 0x55c237213000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.010+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94292636267008] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.010+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.010+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94292636268160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237c00 con 0x55c237213000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.010+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236000 con 0x55c237c8d000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.011+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94292636268160] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.011+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.011+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94292636269312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237c8d800 con 0x55c237c8d000 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.011+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237c8d400 con 0x55c237213c00 2024-04-27T05:21:36.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.012+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.012+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94292636270464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237213400 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.012+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237df3800 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.013+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94292636269312] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.013+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94292636270464] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.013+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.013+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94292636271616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237df3400 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.013+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236c00 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.014+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94292636271616] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94292636272768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237000 con 0x55c237213c00 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=0 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=1 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=2 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=3 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=4 2024-04-27T05:21:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=5 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bfcb20880 20 add_watcher() i=6 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.016+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94292636272768] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.017+0000 7f4bfcb20880 20 add_watcher() i=7 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.017+0000 7f4bfcb20880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.017+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236800 con 0x55c237c8d000 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.017+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.017+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94292636273920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237237400 con 0x55c237c8d000 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94292636273920] v29'92 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4be04f1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bfcb20880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bfcb20880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bfcb20880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bfcb20880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236eb8780 con 0x55c237212c00 2024-04-27T05:21:36.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bf8d22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bf8d22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43458/0 (socket says 172.21.15.138:43458) 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.018+0000 7f4bf8d22640 1 -- 172.21.15.138:0/717913662 learned_addr learned my addr 172.21.15.138:0/717913662 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.019+0000 7f4bf8d22640 1 -- 172.21.15.138:0/717913662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236f06680 con 0x55c237212c00 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.019+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/717913662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55c2371f4930 tx=0x55c2371d6180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d7e1ab4ae30797 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.019+0000 7f4bdfcf0640 1 -- 172.21.15.138:0/717913662 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c237238580 con 0x55c237212c00 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.019+0000 7f4bdfcf0640 1 -- 172.21.15.138:0/717913662 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c237c74000 con 0x55c237212c00 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.019+0000 7f4bdfcf0640 1 -- 172.21.15.138:0/717913662 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2372386e0 con 0x55c237212c00 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- 172.21.15.138:0/717913662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 msgr2=0x55c237178680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/717913662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55c2371f4930 tx=0x55c2371d6180 comp rx=0 tx=0).stop 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- 172.21.15.138:0/717913662 shutdown_connections 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/717913662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- 172.21.15.138:0/717913662 >> 172.21.15.138:0/717913662 conn(0x55c237237800 msgr2=0x55c236f9f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- 172.21.15.138:0/717913662 shutdown_connections 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- 172.21.15.138:0/717913662 wait complete. 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 Processor -- start 2024-04-27T05:21:36.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- start start 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bfcb20880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c236eb8500 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bf8d22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bf8d22640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43472/0 (socket says 172.21.15.138:43472) 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.020+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2803006630 learned_addr learned my addr 172.21.15.138:0/2803006630 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2803006630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c236f064e0 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bf8d22640 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55c2371f5890 tx=0x55c2371d6ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bde4ed640 1 -- 172.21.15.138:0/2803006630 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c237238840 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bde4ed640 1 -- 172.21.15.138:0/2803006630 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c237c74a80 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c236e99380 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bde4ed640 1 -- 172.21.15.138:0/2803006630 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c237238580 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.021+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c236e98ea0 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bde4ed640 1 -- 172.21.15.138:0/2803006630 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c236ebfb80 con 0x55c237212c00 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bde4ed640 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237df3800 0x55c237179700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bde4ed640 1 -- 172.21.15.138:0/2803006630 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c236ea7dc0 con 0x55c237212c00 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bfcb20880 5 note: GC not initialized 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237df3800 0x55c237179700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bdbce8640 20 reqs_thread_entry: start 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bfcb20880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bf9d24640 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237df3800 0x55c237179700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55c2371f5230 tx=0x55c2371d68a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bfcb20880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bd9ce4640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.022+0000 7f4bd94c2640 20 reqs_thread_entry: start 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730c000 con 0x55c237c8d000 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv10 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 20 rados_obj.operate() r=0 bl.length=513 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.023+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730c800 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v29'11 uv11 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bfcb20880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bfcb20880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bfcb20880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94292636276224 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730cc00 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94292636269312 notify 124554051594 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55c2371d5d40 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf3d18640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 94292636269312 notifier 4445 bl.length()=533 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf3d18640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf3d18640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 94292636276224 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.024+0000 7f4bf3d18640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23731c000 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.025+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.025+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94292636276224 notify 124554051594 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c237295a00 con 0x55c237c8d000 2024-04-27T05:21:36.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.025+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730d000 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 foo:foo2 [delete] v29'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bfcb20880 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94292636276224 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730d400 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== watch-notify(notify (1) cookie 94292636264704 notify 124554051589 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x55c237295d40 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf3517640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051589 cookie 94292636264704 notifier 4445 bl.length()=159 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf3517640 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c237236c00 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(67 notify.0 [notify cookie 94292636276224 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.026+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== watch-notify(notify_complete (2) cookie 94292636276224 notify 124554051589 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c237291860 con 0x55c237213000 2024-04-27T05:21:36.040 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:36.041 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:36.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:36.043 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.027+0000 7f4be04f1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.027+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94292636273920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730d800 con 0x55c237c8d000 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.028+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94292636273920] v29'93 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.028+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94292636264704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730dc00 con 0x55c237213000 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.029+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94292636264704] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.029+0000 7f4bfcb20880 20 remove_watcher() i=0 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.029+0000 7f4bfcb20880 2 removed watcher, disabling cache 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.029+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94292636265856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23731c000 con 0x55c237c8d000 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.030+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94292636265856] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.031+0000 7f4bfcb20880 20 remove_watcher() i=1 2024-04-27T05:21:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.031+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94292636267008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23731c800 con 0x55c237213800 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.031+0000 7f4bf9523640 1 -- 172.21.15.138:0/2203927216 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94292636267008] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cd80 con 0x55c237213800 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.031+0000 7f4bfcb20880 20 remove_watcher() i=2 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.031+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94292636268160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730d000 con 0x55c237213000 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.032+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94292636268160] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722cfc0 con 0x55c237213000 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.032+0000 7f4bfcb20880 20 remove_watcher() i=3 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.032+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94292636269312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730d400 con 0x55c237c8d000 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.033+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94292636269312] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237c8d000 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.033+0000 7f4bfcb20880 20 remove_watcher() i=4 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.033+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94292636270464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730dc00 con 0x55c237213c00 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.034+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94292636270464] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.034+0000 7f4bfcb20880 20 remove_watcher() i=5 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.034+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94292636271616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730c000 con 0x55c237213c00 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.035+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94292636271616] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.035+0000 7f4bfcb20880 20 remove_watcher() i=6 2024-04-27T05:21:36.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.035+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94292636272768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c23730c800 con 0x55c237213c00 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bf9d24640 1 -- 172.21.15.138:0/2203927216 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94292636272768] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c23722c6c0 con 0x55c237213c00 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 20 remove_watcher() i=7 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c237213c00 msgr2=0x55c237179180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c237213c00 0x55c237179180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c237c8d000 msgr2=0x55c237177b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c237c8d000 0x55c237177b80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c237213000 msgr2=0x55c237178c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c237213000 0x55c237178c00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c237213800 msgr2=0x55c237178100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c237213800 0x55c237178100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237212000 msgr2=0x55c237177600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237212000 0x55c237177600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x55c237e01260 tx=0x55c2371d6120 comp rx=0 tx=0).stop 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 msgr2=0x55c237176b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2203927216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237c8cc00 0x55c237176b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55c237c995c0 tx=0x55c236fa79e0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 reap_dead start 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 shutdown_connections 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bf8d22640 1 -- 172.21.15.138:0/2203927216 reap_dead start 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.036+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 >> 172.21.15.138:0/2203927216 conn(0x55c236e95400 msgr2=0x55c236f9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 shutdown_connections 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2203927216 wait complete. 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237df3800 msgr2=0x55c237179700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237df3800 0x55c237179700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x55c2371f5230 tx=0x55c2371d68a0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 msgr2=0x55c237178680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55c2371f5890 tx=0x55c2371d6ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 shutdown_connections 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237df3800 0x55c237179700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/2803006630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c237212c00 0x55c237178680 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 >> 172.21.15.138:0/2803006630 conn(0x55c237237800 msgr2=0x55c236f9f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 shutdown_connections 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/2803006630 wait complete. 2024-04-27T05:21:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237c8dc00 msgr2=0x55c237177080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237c8dc00 0x55c237177080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c237d942a0 tx=0x55c236fa6ea0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 msgr2=0x55c237176580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55c237d50de0 tx=0x55c236fa6c60 comp rx=0 tx=0).stop 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 shutdown_connections 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c237c8dc00 0x55c237177080 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 --2- 172.21.15.138:0/7215209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c236e95000 0x55c237176580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 >> 172.21.15.138:0/7215209 conn(0x55c236e94c00 msgr2=0x55c236f9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 shutdown_connections 2024-04-27T05:21:36.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.037+0000 7f4bfcb20880 1 -- 172.21.15.138:0/7215209 wait complete. 2024-04-27T05:21:36.052 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:36.052 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-27T05:21:36.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-27T05:21:36.053 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-27T05:21:36.094 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:36.094 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d4235880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d4235880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a3a8a1e600 con 0x55a3a8747400 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d4235880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a3a8786280 con 0x55a3a8747000 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d01f4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d01f4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43488/0 (socket says 172.21.15.138:43488) 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16d01f4640 1 -- 172.21.15.138:0/3083428985 learned_addr learned my addr 172.21.15.138:0/3083428985 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.093+0000 7f16cf9f3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a3a8747400 0x55a3a884e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60886/0 (socket says 172.21.15.138:60886) 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.094+0000 7f16d01f4640 1 -- 172.21.15.138:0/3083428985 >> v1:172.21.15.114:6789/0 conn(0x55a3a8747400 legacy=0x55a3a884e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.094+0000 7f16d01f4640 1 -- 172.21.15.138:0/3083428985 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a874b380 con 0x55a3a8747000 2024-04-27T05:21:36.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.095+0000 7f16d01f4640 1 --2- 172.21.15.138:0/3083428985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55a3a8852d50 tx=0x55a3a8858de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13920627e15c954a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.095+0000 7f16cf1f2640 1 -- 172.21.15.138:0/3083428985 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875fce0 con 0x55a3a8747000 2024-04-27T05:21:36.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.095+0000 7f16cf1f2640 1 -- 172.21.15.138:0/3083428985 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a8a1fb00 con 0x55a3a8747000 2024-04-27T05:21:36.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.095+0000 7f16cf1f2640 1 -- 172.21.15.138:0/3083428985 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875fe40 con 0x55a3a8747000 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 -- 172.21.15.138:0/3083428985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 msgr2=0x55a3a8a28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 --2- 172.21.15.138:0/3083428985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55a3a8852d50 tx=0x55a3a8858de0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 -- 172.21.15.138:0/3083428985 shutdown_connections 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 --2- 172.21.15.138:0/3083428985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 -- 172.21.15.138:0/3083428985 >> 172.21.15.138:0/3083428985 conn(0x55a3a8746c00 msgr2=0x55a3a884e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 -- 172.21.15.138:0/3083428985 shutdown_connections 2024-04-27T05:21:36.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 -- 172.21.15.138:0/3083428985 wait complete. 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.096+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d4235880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d4235880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a8786780 con 0x55a3a8747000 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d01f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d01f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43498/0 (socket says 172.21.15.138:43498) 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d01f4640 1 -- 172.21.15.138:0/831488535 learned_addr learned my addr 172.21.15.138:0/831488535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d01f4640 1 -- 172.21.15.138:0/831488535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a874aea0 con 0x55a3a8747000 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16d01f4640 1 --2- 172.21.15.138:0/831488535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55a3a954b560 tx=0x55a3a8858cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8f4a46c06e8ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16ce9f1640 1 -- 172.21.15.138:0/831488535 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875f4a0 con 0x55a3a8747000 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16ce9f1640 1 -- 172.21.15.138:0/831488535 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a8a1fe00 con 0x55a3a8747000 2024-04-27T05:21:36.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.097+0000 7f16ce9f1640 1 -- 172.21.15.138:0/831488535 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875fce0 con 0x55a3a8747000 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- 172.21.15.138:0/831488535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 msgr2=0x55a3a8a28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 --2- 172.21.15.138:0/831488535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55a3a954b560 tx=0x55a3a8858cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- 172.21.15.138:0/831488535 shutdown_connections 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 --2- 172.21.15.138:0/831488535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- 172.21.15.138:0/831488535 >> 172.21.15.138:0/831488535 conn(0x55a3a8746c00 msgr2=0x55a3a884f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- 172.21.15.138:0/831488535 shutdown_connections 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- 172.21.15.138:0/831488535 wait complete. 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d4235880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a87868c0 con 0x55a3a8747000 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d01f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d01f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43512/0 (socket says 172.21.15.138:43512) 2024-04-27T05:21:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.098+0000 7f16d01f4640 1 -- 172.21.15.138:0/286775105 learned_addr learned my addr 172.21.15.138:0/286775105 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16d01f4640 1 -- 172.21.15.138:0/286775105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a874b6c0 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16d01f4640 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55a3a96032c0 tx=0x55a3a8858c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16cd1ee640 1 -- 172.21.15.138:0/286775105 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875fe40 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16cd1ee640 1 -- 172.21.15.138:0/286775105 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a9526000 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a3a874b1e0 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16cd1ee640 1 -- 172.21.15.138:0/286775105 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875f4a0 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.099+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3a87b8ea0 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16cd1ee640 1 -- 172.21.15.138:0/286775105 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a3a8770b00 con 0x55a3a8747000 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d4235880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d4235880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a8786c80 con 0x55a3a953ec00 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16cd1ee640 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a953fc00 0x55a3a8a29080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16cd1ee640 1 -- 172.21.15.138:0/286775105 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a3a8759a40 con 0x55a3a8747000 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d01f4640 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a953fc00 0x55a3a8a29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d09f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d09f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43528/0 (socket says 172.21.15.138:43528) 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d09f5640 1 -- 172.21.15.138:0/1837030875 learned_addr learned my addr 172.21.15.138:0/1837030875 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d09f5640 1 -- 172.21.15.138:0/1837030875 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a87b8d00 con 0x55a3a953ec00 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d01f4640 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a953fc00 0x55a3a8a29080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55a3a9648180 tx=0x55a3a8858cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.100+0000 7f16d09f5640 1 --2- 172.21.15.138:0/1837030875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55a3a96035f0 tx=0x55a3a8859c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3187671c9b157da server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16ca9e9640 1 -- 172.21.15.138:0/1837030875 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875fe40 con 0x55a3a953ec00 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16ca9e9640 1 -- 172.21.15.138:0/1837030875 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a9526600 con 0x55a3a953ec00 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16ca9e9640 1 -- 172.21.15.138:0/1837030875 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875f4a0 con 0x55a3a953ec00 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- 172.21.15.138:0/1837030875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 msgr2=0x55a3a8a28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 --2- 172.21.15.138:0/1837030875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55a3a96035f0 tx=0x55a3a8859c20 comp rx=0 tx=0).stop 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- 172.21.15.138:0/1837030875 shutdown_connections 2024-04-27T05:21:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 --2- 172.21.15.138:0/1837030875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- 172.21.15.138:0/1837030875 >> 172.21.15.138:0/1837030875 conn(0x55a3a8747400 msgr2=0x55a3a8851000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- 172.21.15.138:0/1837030875 shutdown_connections 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- 172.21.15.138:0/1837030875 wait complete. 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.101+0000 7f16d4235880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a876a280 con 0x55a3a953ec00 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d09f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d09f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43542/0 (socket says 172.21.15.138:43542) 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 learned_addr learned my addr 172.21.15.138:0/3073035116 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a87b8b60 con 0x55a3a953ec00 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d09f5640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55a3a96033b0 tx=0x55a3a8859980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16c91e6640 1 -- 172.21.15.138:0/3073035116 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875fe40 con 0x55a3a953ec00 2024-04-27T05:21:36.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16c91e6640 1 -- 172.21.15.138:0/3073035116 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a9526780 con 0x55a3a953ec00 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16c91e6640 1 -- 172.21.15.138:0/3073035116 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a875f4a0 con 0x55a3a953ec00 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a3a87b89c0 con 0x55a3a953ec00 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.102+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3a87b8820 con 0x55a3a953ec00 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16c91e6640 1 -- 172.21.15.138:0/3073035116 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a3a8771600 con 0x55a3a953ec00 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16c91e6640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a8ac0000 0x55a3a8a29600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16c91e6640 1 -- 172.21.15.138:0/3073035116 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a3a8759c00 con 0x55a3a953ec00 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a8ac0000 0x55a3a8a29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a3a953f000 0x55a3a8a29b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a953f400 con 0x55a3a953f000 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.103+0000 7f16d09f5640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a3a953f000 0x55a3a8a29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a8ac0000 0x55a3a8a29600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55a3a8853380 tx=0x55a3a8a86120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d09f5640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a3a953f000 0x55a3a8a29b80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 realm 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a953f800 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1000 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.104+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1400 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a3a8ac1800 0x55a3a8a2a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d01f4640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a3a8ac1800 0x55a3a8a2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d01f4640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a3a8ac1800 0x55a3a8a2a100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 20 searching for the correct realm 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.105+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1000 con 0x55a3a8ac1800 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16d01f4640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a3a8ac1000 0x55a3a8a2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a3a8ac1000 0x55a3a8a2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a3a8ac1000 0x55a3a8a2ac00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a3a8ac1c00 0x55a3a8a2b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.106+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a3a8ac1c00 0x55a3a8a2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a3a8ac1c00 0x55a3a8a2b180 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.107+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.108+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.109+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.110+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.110+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.110+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.110+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.110+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.110+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.111+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.112+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a953f000 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.113+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac0c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:36.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1400 con 0x55a3a953f000 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a953f400 con 0x55a3a953f000 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a953f800 con 0x55a3a8ac1800 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 zone default found 2024-04-27T05:21:36.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 4 Realm: () 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 zonegroup default 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.114+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8000 con 0x55a3a8ac1c00 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8400 con 0x55a3a8ac1800 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8800 con 0x55a3a8ac1800 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 started sync module instance, tier type = 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.115+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.116+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.116+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94161397221632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9000 con 0x55a3a8ac1000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.116+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9400 con 0x55a3a953f000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.117+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94161397221632] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.117+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.118+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94161397222784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9800 con 0x55a3a953f000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.118+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9c00 con 0x55a3a8ac1800 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.119+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94161397222784] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.119+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.119+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94161397223936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8400 con 0x55a3a8ac1800 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.119+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8800 con 0x55a3a8ac1000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.120+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.120+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94161397225088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9c00 con 0x55a3a8ac1000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.120+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8000 con 0x55a3a953f000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.121+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94161397225088] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.121+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94161397223936] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.121+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.121+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94161397226240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a953f800 con 0x55a3a953f000 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.121+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a953f400 con 0x55a3a8ac1c00 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.122+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.122+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94161397227392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ac1400 con 0x55a3a8ac1c00 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.122+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a96a4c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.122+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94161397226240] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.123+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.123+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94161397228544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a96a4800 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.123+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8c00 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.123+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94161397227392] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94161397228544] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94161397229696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9000 con 0x55a3a8ac1c00 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=0 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=1 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=2 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=3 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=4 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=5 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.124+0000 7f16d4235880 20 add_watcher() i=6 2024-04-27T05:21:36.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.125+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94161397229696] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.125+0000 7f16d4235880 20 add_watcher() i=7 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.125+0000 7f16d4235880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.125+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae8800 con 0x55a3a953f000 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.126+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.126+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94161397230848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8ae9400 con 0x55a3a953f000 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.126+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94161397230848] v29'94 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.126+0000 7f16b71c2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16d4235880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16d4235880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16d4235880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16d4235880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a876a8c0 con 0x55a3a8ac0c00 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16cf9f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16cf9f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43550/0 (socket says 172.21.15.138:43550) 2024-04-27T05:21:36.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16cf9f3640 1 -- 172.21.15.138:0/924750846 learned_addr learned my addr 172.21.15.138:0/924750846 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16cf9f3640 1 -- 172.21.15.138:0/924750846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a87b8680 con 0x55a3a8ac0c00 2024-04-27T05:21:36.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.127+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/924750846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55a3a8aa2ae0 tx=0x55a3a8a86180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d9007a05fbedb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16b69c1640 1 -- 172.21.15.138:0/924750846 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a8aea580 con 0x55a3a8ac0c00 2024-04-27T05:21:36.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16b69c1640 1 -- 172.21.15.138:0/924750846 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a9526000 con 0x55a3a8ac0c00 2024-04-27T05:21:36.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16b69c1640 1 -- 172.21.15.138:0/924750846 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a8aea6e0 con 0x55a3a8ac0c00 2024-04-27T05:21:36.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 -- 172.21.15.138:0/924750846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 msgr2=0x55a3a8a2a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 --2- 172.21.15.138:0/924750846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55a3a8aa2ae0 tx=0x55a3a8a86180 comp rx=0 tx=0).stop 2024-04-27T05:21:36.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 -- 172.21.15.138:0/924750846 shutdown_connections 2024-04-27T05:21:36.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 --2- 172.21.15.138:0/924750846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 -- 172.21.15.138:0/924750846 >> 172.21.15.138:0/924750846 conn(0x55a3a8ae9800 msgr2=0x55a3a8851800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 -- 172.21.15.138:0/924750846 shutdown_connections 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.128+0000 7f16d4235880 1 -- 172.21.15.138:0/924750846 wait complete. 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16d4235880 1 Processor -- start 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16d4235880 1 -- start start 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16d4235880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16d4235880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3a876a780 con 0x55a3a8ac0c00 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16cf9f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16cf9f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43552/0 (socket says 172.21.15.138:43552) 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16cf9f3640 1 -- 172.21.15.138:0/2403228261 learned_addr learned my addr 172.21.15.138:0/2403228261 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16cf9f3640 1 -- 172.21.15.138:0/2403228261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3a87b84e0 con 0x55a3a8ac0c00 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16cf9f3640 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a3a8aa3860 tx=0x55a3a8a86ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16b51be640 1 -- 172.21.15.138:0/2403228261 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a8aea840 con 0x55a3a8ac0c00 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.129+0000 7f16b51be640 1 -- 172.21.15.138:0/2403228261 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3a9526a80 con 0x55a3a8ac0c00 2024-04-27T05:21:36.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a3a874b380 con 0x55a3a8ac0c00 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16b51be640 1 -- 172.21.15.138:0/2403228261 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a3a8aea580 con 0x55a3a8ac0c00 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3a874aea0 con 0x55a3a8ac0c00 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16b51be640 1 -- 172.21.15.138:0/2403228261 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a3a8771b80 con 0x55a3a8ac0c00 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16b51be640 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a96a4c00 0x55a3a8a2b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16b51be640 1 -- 172.21.15.138:0/2403228261 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a3a8759dc0 con 0x55a3a8ac0c00 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16d4235880 5 note: GC not initialized 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.130+0000 7f16d09f5640 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a96a4c00 0x55a3a8a2b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16b29b9640 20 reqs_thread_entry: start 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16d4235880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16d09f5640 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a96a4c00 0x55a3a8a2b700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55a3a96aec30 tx=0x55a3a8a868a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16d4235880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16b09b5640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:36.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16b0193640 20 reqs_thread_entry: start 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16d4235880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bba000 con 0x55a3a953f000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.131+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.132+0000 7f16d4235880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.132+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bba400 con 0x55a3a8ac1000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.132+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.132+0000 7f16b71c2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.132+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94161397230848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bba800 con 0x55a3a953f000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.133+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94161397230848] v29'95 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.133+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94161397221632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bbac00 con 0x55a3a8ac1000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.134+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94161397221632] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.134+0000 7f16d4235880 20 remove_watcher() i=0 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.135+0000 7f16d4235880 2 removed watcher, disabling cache 2024-04-27T05:21:36.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.135+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94161397222784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bbb000 con 0x55a3a953f000 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.135+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94161397222784] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.136+0000 7f16d4235880 20 remove_watcher() i=1 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.136+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94161397223936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bbb400 con 0x55a3a8ac1800 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.136+0000 7f16d01f4640 1 -- 172.21.15.138:0/3073035116 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94161397223936] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adafc0 con 0x55a3a8ac1800 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.136+0000 7f16d4235880 20 remove_watcher() i=2 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.136+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94161397225088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bbb800 con 0x55a3a8ac1000 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.137+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94161397225088] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8adb200 con 0x55a3a8ac1000 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.137+0000 7f16d4235880 20 remove_watcher() i=3 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.137+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94161397226240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8bbbc00 con 0x55a3a953f000 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.138+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94161397226240] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a953f000 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.138+0000 7f16d4235880 20 remove_watcher() i=4 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.138+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94161397227392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8a70000 con 0x55a3a8ac1c00 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.139+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94161397227392] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.139+0000 7f16d4235880 20 remove_watcher() i=5 2024-04-27T05:21:36.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.139+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94161397228544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8a70400 con 0x55a3a8ac1c00 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.140+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94161397228544] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.140+0000 7f16d4235880 20 remove_watcher() i=6 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.140+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94161397229696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a3a8a70800 con 0x55a3a8ac1c00 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d09f5640 1 -- 172.21.15.138:0/3073035116 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94161397229696] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a3a8ada900 con 0x55a3a8ac1c00 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 20 remove_watcher() i=7 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a3a8ac1c00 msgr2=0x55a3a8a2b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a3a8ac1c00 0x55a3a8a2b180 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a3a953f000 msgr2=0x55a3a8a29b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a3a953f000 0x55a3a8a29b80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a3a8ac1000 msgr2=0x55a3a8a2ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a3a8ac1000 0x55a3a8a2ac00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a3a8ac1800 msgr2=0x55a3a8a2a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a3a8ac1800 0x55a3a8a2a100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a8ac0000 msgr2=0x55a3a8a29600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a8ac0000 0x55a3a8a29600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55a3a8853380 tx=0x55a3a8a86120 comp rx=0 tx=0).stop 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 msgr2=0x55a3a8a28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/3073035116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a953ec00 0x55a3a8a28b00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55a3a96033b0 tx=0x55a3a8859980 comp rx=0 tx=0).stop 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 reap_dead start 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16cf9f3640 1 -- 172.21.15.138:0/3073035116 reap_dead start 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 shutdown_connections 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 >> 172.21.15.138:0/3073035116 conn(0x55a3a8747400 msgr2=0x55a3a8851000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 shutdown_connections 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/3073035116 wait complete. 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a96a4c00 msgr2=0x55a3a8a2b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a96a4c00 0x55a3a8a2b700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55a3a96aec30 tx=0x55a3a8a868a0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 msgr2=0x55a3a8a2a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.141+0000 7f16d4235880 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a3a8aa3860 tx=0x55a3a8a86ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 shutdown_connections 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a96a4c00 0x55a3a8a2b700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 --2- 172.21.15.138:0/2403228261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8ac0c00 0x55a3a8a2a680 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 >> 172.21.15.138:0/2403228261 conn(0x55a3a8ae9800 msgr2=0x55a3a8851800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 shutdown_connections 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/2403228261 wait complete. 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a953fc00 msgr2=0x55a3a8a29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a953fc00 0x55a3a8a29080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55a3a9648180 tx=0x55a3a8858cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 msgr2=0x55a3a8a28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55a3a96032c0 tx=0x55a3a8858c60 comp rx=0 tx=0).stop 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 shutdown_connections 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a3a953fc00 0x55a3a8a29080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 --2- 172.21.15.138:0/286775105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3a8747000 0x55a3a8a28580 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 >> 172.21.15.138:0/286775105 conn(0x55a3a8746c00 msgr2=0x55a3a884f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 shutdown_connections 2024-04-27T05:21:36.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.142+0000 7f16d4235880 1 -- 172.21.15.138:0/286775105 wait complete. 2024-04-27T05:21:36.157 INFO:teuthology.orchestra.run.smithi138.stdout:[] 2024-04-27T05:21:36.157 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T05:21:36.158 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-27T05:21:36.158 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-27T05:21:36.158 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-27T05:21:36.241 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:36.241 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.239+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.239+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.239+0000 7f4caa20f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5619fc8d5400 0x5619fcbb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.239+0000 7f4caa20f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5619fcbac600 con 0x5619fc8d5000 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.239+0000 7f4caa20f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5619fc914280 con 0x5619fc8d5400 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.240+0000 7f4ca5fda640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5619fc8d5400 0x5619fcbb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.240+0000 7f4ca5fda640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5619fc8d5400 0x5619fcbb6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43564/0 (socket says 172.21.15.138:43564) 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.240+0000 7f4ca5fda640 1 -- 172.21.15.138:0/294503680 learned_addr learned my addr 172.21.15.138:0/294503680 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.240+0000 7f4ca67db640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5619fc8d5000 0x5619fc9dc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60890/0 (socket says 172.21.15.138:60890) 2024-04-27T05:21:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.240+0000 7f4ca5fda640 1 -- 172.21.15.138:0/294503680 >> v1:172.21.15.114:6789/0 conn(0x5619fc8d5000 legacy=0x5619fc9dc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.240+0000 7f4ca5fda640 1 -- 172.21.15.138:0/294503680 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc8d9380 con 0x5619fc8d5400 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.241+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/294503680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5400 0x5619fcbb6580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5619fc9e0c00 tx=0x5619fc9e6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d2cce2b5f17b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4ca57d9640 1 -- 172.21.15.138:0/294503680 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8edce0 con 0x5619fc8d5400 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4ca57d9640 1 -- 172.21.15.138:0/294503680 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fcbadb00 con 0x5619fc8d5400 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4ca57d9640 1 -- 172.21.15.138:0/294503680 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ede40 con 0x5619fc8d5400 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 -- 172.21.15.138:0/294503680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5400 msgr2=0x5619fcbb6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 --2- 172.21.15.138:0/294503680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5400 0x5619fcbb6580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5619fc9e0c00 tx=0x5619fc9e6de0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 -- 172.21.15.138:0/294503680 shutdown_connections 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 --2- 172.21.15.138:0/294503680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5400 0x5619fcbb6580 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 -- 172.21.15.138:0/294503680 >> 172.21.15.138:0/294503680 conn(0x5619fc8d4c00 msgr2=0x5619fc9dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 -- 172.21.15.138:0/294503680 shutdown_connections 2024-04-27T05:21:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.242+0000 7f4caa20f880 1 -- 172.21.15.138:0/294503680 wait complete. 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4caa20f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4caa20f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619fc914780 con 0x5619fc8d5000 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4ca67db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4ca67db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43578/0 (socket says 172.21.15.138:43578) 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.243+0000 7f4ca67db640 1 -- 172.21.15.138:0/2872219277 learned_addr learned my addr 172.21.15.138:0/2872219277 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4ca67db640 1 -- 172.21.15.138:0/2872219277 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc8d8ea0 con 0x5619fc8d5000 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4ca67db640 1 --2- 172.21.15.138:0/2872219277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5619fd791290 tx=0x5619fc9e6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70353cf52d96b929 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4ca4fd8640 1 -- 172.21.15.138:0/2872219277 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ed4a0 con 0x5619fc8d5000 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4ca4fd8640 1 -- 172.21.15.138:0/2872219277 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fcbade00 con 0x5619fc8d5000 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4ca4fd8640 1 -- 172.21.15.138:0/2872219277 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8edce0 con 0x5619fc8d5000 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 -- 172.21.15.138:0/2872219277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 msgr2=0x5619fcbb6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 --2- 172.21.15.138:0/2872219277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5619fd791290 tx=0x5619fc9e6cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 -- 172.21.15.138:0/2872219277 shutdown_connections 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 --2- 172.21.15.138:0/2872219277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 -- 172.21.15.138:0/2872219277 >> 172.21.15.138:0/2872219277 conn(0x5619fc8d4c00 msgr2=0x5619fc9dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 -- 172.21.15.138:0/2872219277 shutdown_connections 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.244+0000 7f4caa20f880 1 -- 172.21.15.138:0/2872219277 wait complete. 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4caa20f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4caa20f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619fc9148c0 con 0x5619fc8d5000 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca67db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca67db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43590/0 (socket says 172.21.15.138:43590) 2024-04-27T05:21:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca67db640 1 -- 172.21.15.138:0/2030562664 learned_addr learned my addr 172.21.15.138:0/2030562664 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca67db640 1 -- 172.21.15.138:0/2030562664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc8d96c0 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca67db640 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5619fd790b40 tx=0x5619fc9e6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca37d5640 1 -- 172.21.15.138:0/2030562664 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ede40 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca37d5640 1 -- 172.21.15.138:0/2030562664 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fd7a6000 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619fc8d91e0 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.245+0000 7f4ca37d5640 1 -- 172.21.15.138:0/2030562664 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ed4a0 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619fc946ea0 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4caa20f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4caa20f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619fc914c80 con 0x5619fd6cd800 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca37d5640 1 -- 172.21.15.138:0/2030562664 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5619fc8ff600 con 0x5619fc8d5000 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca6fdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca37d5640 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd834000 0x5619fcbb7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca6fdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43602/0 (socket says 172.21.15.138:43602) 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/489064958 learned_addr learned my addr 172.21.15.138:0/489064958 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca37d5640 1 -- 172.21.15.138:0/2030562664 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5619fc8e7a40 con 0x5619fc8d5000 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.246+0000 7f4ca67db640 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd834000 0x5619fcbb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/489064958 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc946d00 con 0x5619fd6cd800 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/489064958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x5619fd7915f0 tx=0x5619fc9e7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342c984366cf46d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4ca67db640 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd834000 0x5619fcbb7080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5619fd7d6a50 tx=0x5619fc9e74a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4ca0fd0640 1 -- 172.21.15.138:0/489064958 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ede40 con 0x5619fd6cd800 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4ca0fd0640 1 -- 172.21.15.138:0/489064958 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fd7a6300 con 0x5619fd6cd800 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4ca0fd0640 1 -- 172.21.15.138:0/489064958 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ed4a0 con 0x5619fd6cd800 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 -- 172.21.15.138:0/489064958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 msgr2=0x5619fcbb6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 --2- 172.21.15.138:0/489064958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x5619fd7915f0 tx=0x5619fc9e7a40 comp rx=0 tx=0).stop 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 -- 172.21.15.138:0/489064958 shutdown_connections 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 --2- 172.21.15.138:0/489064958 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 -- 172.21.15.138:0/489064958 >> 172.21.15.138:0/489064958 conn(0x5619fc8d5400 msgr2=0x5619fc9df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 -- 172.21.15.138:0/489064958 shutdown_connections 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.247+0000 7f4caa20f880 1 -- 172.21.15.138:0/489064958 wait complete. 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4caa20f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4caa20f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619fc8f8280 con 0x5619fd6cd800 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4ca6fdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4ca6fdc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43616/0 (socket says 172.21.15.138:43616) 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 learned_addr learned my addr 172.21.15.138:0/1257171048 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc946b60 con 0x5619fd6cd800 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.248+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5619fd791a70 tx=0x5619fc9e7aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4c9f7cd640 1 -- 172.21.15.138:0/1257171048 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ede40 con 0x5619fd6cd800 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4c9f7cd640 1 -- 172.21.15.138:0/1257171048 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fd7a6480 con 0x5619fd6cd800 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4c9f7cd640 1 -- 172.21.15.138:0/1257171048 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fc8ed4a0 con 0x5619fd6cd800 2024-04-27T05:21:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619fc9469c0 con 0x5619fd6cd800 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619fc946820 con 0x5619fd6cd800 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4c9f7cd640 1 -- 172.21.15.138:0/1257171048 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5619fc8ff8c0 con 0x5619fd6cd800 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.249+0000 7f4c9f7cd640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fcc5e000 0x5619fcbb7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4c9f7cd640 1 -- 172.21.15.138:0/1257171048 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5619fc8e7c00 con 0x5619fd6cd800 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fcc5e000 0x5619fcbb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5619fd6cdc00 0x5619fcbb7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5e400 con 0x5619fd6cdc00 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5619fd6cdc00 0x5619fcbb7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fcc5e000 0x5619fcbb7600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5619fc9e0c00 tx=0x5619fcc20120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5619fd6cdc00 0x5619fcbb7b80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4caa20f880 20 realm 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5e800 con 0x5619fd6cdc00 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.250+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5ec00 con 0x5619fd6cdc00 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5f000 con 0x5619fd6cdc00 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619fcc5f400 0x5619fcbb8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5e400 con 0x5619fcc5f400 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4ca67db640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619fcc5f400 0x5619fcbb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.251+0000 7f4ca67db640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619fcc5f400 0x5619fcbb8100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4caa20f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4caa20f880 20 searching for the correct realm 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5e800 con 0x5619fcc5f400 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4ca67db640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619fcc5e800 0x5619fcbb8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5e400 con 0x5619fcc5e800 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619fcc5e800 0x5619fcbb8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.252+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619fcc5e800 0x5619fcbb8c00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619fcc5e400 0x5619fcbb9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619fcc5e400 0x5619fcbb9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619fcc5e400 0x5619fcbb9180 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.253+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e800 2024-04-27T05:21:36.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.254+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e800 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.255+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.256+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.257+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e800 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.258+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e800 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5f400 2024-04-27T05:21:36.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e400 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fd6cdc00 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.259+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6cc800 con 0x5619fcc5e800 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5ec00 con 0x5619fd6cdc00 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5f000 con 0x5619fd6cdc00 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5f800 con 0x5619fcc5f400 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 zone default found 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 4 Realm: () 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:36.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 zonegroup default 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.260+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5fc00 con 0x5619fcc5e400 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8000 con 0x5619fcc5f400 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8400 con 0x5619fcc5f400 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 started sync module instance, tier type = 2024-04-27T05:21:36.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.261+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8800 con 0x5619fcc5e800 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.262+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.262+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94669614278912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8c00 con 0x5619fcc5e800 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.262+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b9000 con 0x5619fd6cdc00 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.263+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94669614278912] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.264+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.264+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94669614280064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b9400 con 0x5619fd6cdc00 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.264+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b9800 con 0x5619fcc5f400 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.265+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.265+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94669614281216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8000 con 0x5619fcc5f400 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.265+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8400 con 0x5619fcc5e800 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.265+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94669614280064] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.266+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.266+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94669614281216] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.266+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94669614282368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b9800 con 0x5619fcc5e800 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.266+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5f800 con 0x5619fd6cdc00 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.267+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.267+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94669614283520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5ec00 con 0x5619fd6cdc00 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.267+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94669614282368] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.267+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5f000 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94669614284672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcc5fc00 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd835800 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94669614283520] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94669614284672] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.268+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94669614285824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8800 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8c00 con 0x5619fcc5e400 2024-04-27T05:21:36.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94669614286976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b8400 con 0x5619fcc5e400 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=0 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=1 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=2 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=3 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=4 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=5 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94669614285824] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.269+0000 7f4caa20f880 20 add_watcher() i=6 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.270+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94669614286976] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.270+0000 7f4caa20f880 20 add_watcher() i=7 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.270+0000 7f4caa20f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.270+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b9800 con 0x5619fd6cdc00 2024-04-27T05:21:36.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.271+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.271+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94669614288128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fd6b9000 con 0x5619fd6cdc00 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.271+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94669614288128] v29'96 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4c8d7a9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4caa20f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4caa20f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4caa20f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4caa20f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619fc8f88c0 con 0x5619fd6b9c00 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4ca5fda640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4ca5fda640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43618/0 (socket says 172.21.15.138:43618) 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.272+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1586518539 learned_addr learned my addr 172.21.15.138:0/1586518539 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1586518539 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc946680 con 0x5619fd6b9c00 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/1586518539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5619fcc38b10 tx=0x5619fcc200c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccd41679bd0f583 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4c8cfa8640 1 -- 172.21.15.138:0/1586518539 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fd6ba580 con 0x5619fd6b9c00 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4c8cfa8640 1 -- 172.21.15.138:0/1586518539 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fd7a6000 con 0x5619fd6b9c00 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4c8cfa8640 1 -- 172.21.15.138:0/1586518539 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fd6ba6e0 con 0x5619fd6b9c00 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4caa20f880 1 -- 172.21.15.138:0/1586518539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 msgr2=0x5619fcbb8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.273+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1586518539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x5619fcc38b10 tx=0x5619fcc200c0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 -- 172.21.15.138:0/1586518539 shutdown_connections 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1586518539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 -- 172.21.15.138:0/1586518539 >> 172.21.15.138:0/1586518539 conn(0x5619fd6b9400 msgr2=0x5619fc9df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 -- 172.21.15.138:0/1586518539 shutdown_connections 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 -- 172.21.15.138:0/1586518539 wait complete. 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 Processor -- start 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 -- start start 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4caa20f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619fc8f8780 con 0x5619fd6b9c00 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4ca5fda640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4ca5fda640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43624/0 (socket says 172.21.15.138:43624) 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.274+0000 7f4ca5fda640 1 -- 172.21.15.138:0/110513331 learned_addr learned my addr 172.21.15.138:0/110513331 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4ca5fda640 1 -- 172.21.15.138:0/110513331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619fc9464e0 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4ca5fda640 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5619fcc39800 tx=0x5619fcc20000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4c8b7a5640 1 -- 172.21.15.138:0/110513331 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fd6ba840 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4c8b7a5640 1 -- 172.21.15.138:0/110513331 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619fd7a6780 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4c8b7a5640 1 -- 172.21.15.138:0/110513331 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619fd6ba580 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619fc8d8ea0 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.275+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619fc8d96c0 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4c8b7a5640 1 -- 172.21.15.138:0/110513331 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5619fc8feb00 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4c8b7a5640 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd6b8800 0x5619fcbb9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4c8b7a5640 1 -- 172.21.15.138:0/110513331 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5619fc8e7dc0 con 0x5619fd6b9c00 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4caa20f880 5 note: GC not initialized 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd6b8800 0x5619fcbb9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4c88fa0640 20 reqs_thread_entry: start 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4caa20f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4ca6fdc640 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd6b8800 0x5619fcbb9700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5619fd84ad20 tx=0x5619fcc20960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4caa20f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4c86f9c640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4c8677a640 20 reqs_thread_entry: start 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4caa20f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.276+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4a000 con 0x5619fd6cdc00 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.277+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.277+0000 7f4caa20f880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.277+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4a400 con 0x5619fcc5e800 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.277+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.278+0000 7f4c8d7a9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.278+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94669614288128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4a800 con 0x5619fd6cdc00 2024-04-27T05:21:36.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.279+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94669614288128] v29'97 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.279+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94669614278912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4ac00 con 0x5619fcc5e800 2024-04-27T05:21:36.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.280+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94669614278912] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.280+0000 7f4caa20f880 20 remove_watcher() i=0 2024-04-27T05:21:36.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.280+0000 7f4caa20f880 2 removed watcher, disabling cache 2024-04-27T05:21:36.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.280+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94669614280064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4b000 con 0x5619fd6cdc00 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.281+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94669614280064] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.281+0000 7f4caa20f880 20 remove_watcher() i=1 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.281+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94669614281216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4b400 con 0x5619fcc5f400 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.282+0000 7f4ca67db640 1 -- 172.21.15.138:0/1257171048 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94669614281216] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6cfc0 con 0x5619fcc5f400 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.282+0000 7f4caa20f880 20 remove_watcher() i=2 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.282+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94669614282368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4b800 con 0x5619fcc5e800 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.282+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94669614282368] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6d200 con 0x5619fcc5e800 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.283+0000 7f4caa20f880 20 remove_watcher() i=3 2024-04-27T05:21:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.283+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94669614283520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcd4bc00 con 0x5619fd6cdc00 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.283+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94669614283520] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fd6cdc00 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.283+0000 7f4caa20f880 20 remove_watcher() i=4 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.283+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94669614284672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcbfc000 con 0x5619fcc5e400 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.284+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94669614284672] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.284+0000 7f4caa20f880 20 remove_watcher() i=5 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.284+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94669614285824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcbfc400 con 0x5619fcc5e400 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.285+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94669614285824] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.285+0000 7f4caa20f880 20 remove_watcher() i=6 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.285+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94669614286976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619fcbfc800 con 0x5619fcc5e400 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4ca6fdc640 1 -- 172.21.15.138:0/1257171048 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94669614286976] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619fcc6c900 con 0x5619fcc5e400 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 20 remove_watcher() i=7 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619fcc5e400 msgr2=0x5619fcbb9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619fcc5e400 0x5619fcbb9180 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5619fd6cdc00 msgr2=0x5619fcbb7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5619fd6cdc00 0x5619fcbb7b80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619fcc5e800 msgr2=0x5619fcbb8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619fcc5e800 0x5619fcbb8c00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619fcc5f400 msgr2=0x5619fcbb8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619fcc5f400 0x5619fcbb8100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fcc5e000 msgr2=0x5619fcbb7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fcc5e000 0x5619fcbb7600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5619fc9e0c00 tx=0x5619fcc20120 comp rx=0 tx=0).stop 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 msgr2=0x5619fcbb6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5619fd791a70 tx=0x5619fc9e7aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4ca5fda640 1 -- 172.21.15.138:0/1257171048 reap_dead start 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 shutdown_connections 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 --2- 172.21.15.138:0/1257171048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6cd800 0x5619fcbb6b00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 >> 172.21.15.138:0/1257171048 conn(0x5619fc8d5400 msgr2=0x5619fc9df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 shutdown_connections 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.286+0000 7f4caa20f880 1 -- 172.21.15.138:0/1257171048 wait complete. 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd6b8800 msgr2=0x5619fcbb9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd6b8800 0x5619fcbb9700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5619fd84ad20 tx=0x5619fcc20960 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 msgr2=0x5619fcbb8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5619fcc39800 tx=0x5619fcc20000 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 shutdown_connections 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd6b8800 0x5619fcbb9700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/110513331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fd6b9c00 0x5619fcbb8680 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 >> 172.21.15.138:0/110513331 conn(0x5619fd6b9400 msgr2=0x5619fc9df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 shutdown_connections 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/110513331 wait complete. 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd834000 msgr2=0x5619fcbb7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd834000 0x5619fcbb7080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5619fd7d6a50 tx=0x5619fc9e74a0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 msgr2=0x5619fcbb6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5619fd790b40 tx=0x5619fc9e6c60 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 shutdown_connections 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619fd834000 0x5619fcbb7080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 --2- 172.21.15.138:0/2030562664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619fc8d5000 0x5619fcbb6580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 >> 172.21.15.138:0/2030562664 conn(0x5619fc8d4c00 msgr2=0x5619fc9dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 shutdown_connections 2024-04-27T05:21:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.287+0000 7f4caa20f880 1 -- 172.21.15.138:0/2030562664 wait complete. 2024-04-27T05:21:36.304 INFO:teuthology.orchestra.run.smithi138.stdout:[] 2024-04-27T05:21:36.305 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T05:21:36.338 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.338 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-27T05:21:36.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-27T05:21:36.338 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-27T05:21:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7b0662880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7b0662880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a1094de600 con 0x55a109207400 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7b0662880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a109246280 con 0x55a109207000 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7aee07640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7ae606640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a109207400 0x55a10930e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60902/0 (socket says 172.21.15.138:60902) 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7aee07640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43632/0 (socket says 172.21.15.138:43632) 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7ae606640 1 -- 172.21.15.138:0/4126743414 learned_addr learned my addr 172.21.15.138:0/4126743414 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7aee07640 1 -- 172.21.15.138:0/4126743414 >> v1:172.21.15.114:6789/0 conn(0x55a109207400 legacy=0x55a10930e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7aee07640 1 -- 172.21.15.138:0/4126743414 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a10920b380 con 0x55a109207000 2024-04-27T05:21:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.387+0000 7fa7ade05640 1 -- 172.21.15.138:0/4126743414 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862519642 0 0) 0x55a10920a9c0 con 0x55a109207400 2024-04-27T05:21:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.389+0000 7fa7aee07640 1 --2- 172.21.15.138:0/4126743414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55a109312d50 tx=0x55a109318de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550c2b8dcfcf9ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.389+0000 7fa7ade05640 1 -- 172.21.15.138:0/4126743414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921fce0 con 0x55a109207000 2024-04-27T05:21:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.389+0000 7fa7ade05640 1 -- 172.21.15.138:0/4126743414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1094dfb00 con 0x55a109207000 2024-04-27T05:21:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.389+0000 7fa7ade05640 1 -- 172.21.15.138:0/4126743414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921fe40 con 0x55a109207000 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 -- 172.21.15.138:0/4126743414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 msgr2=0x55a1094e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 --2- 172.21.15.138:0/4126743414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55a109312d50 tx=0x55a109318de0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 -- 172.21.15.138:0/4126743414 shutdown_connections 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 --2- 172.21.15.138:0/4126743414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 -- 172.21.15.138:0/4126743414 >> 172.21.15.138:0/4126743414 conn(0x55a109206c00 msgr2=0x55a10930e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 -- 172.21.15.138:0/4126743414 shutdown_connections 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 -- 172.21.15.138:0/4126743414 wait complete. 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.390+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7b0662880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7b0662880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109246780 con 0x55a109207000 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7aee07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7aee07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43646/0 (socket says 172.21.15.138:43646) 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7aee07640 1 -- 172.21.15.138:0/1851165790 learned_addr learned my addr 172.21.15.138:0/1851165790 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7aee07640 1 -- 172.21.15.138:0/1851165790 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a10920aea0 con 0x55a109207000 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7aee07640 1 --2- 172.21.15.138:0/1851165790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55a10a00d590 tx=0x55a109318cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e699dfefe6164a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7ad604640 1 -- 172.21.15.138:0/1851165790 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921f4a0 con 0x55a109207000 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7ad604640 1 -- 172.21.15.138:0/1851165790 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1094dfe00 con 0x55a109207000 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.391+0000 7fa7ad604640 1 -- 172.21.15.138:0/1851165790 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921fce0 con 0x55a109207000 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- 172.21.15.138:0/1851165790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 msgr2=0x55a1094e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 --2- 172.21.15.138:0/1851165790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55a10a00d590 tx=0x55a109318cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- 172.21.15.138:0/1851165790 shutdown_connections 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 --2- 172.21.15.138:0/1851165790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- 172.21.15.138:0/1851165790 >> 172.21.15.138:0/1851165790 conn(0x55a109206c00 msgr2=0x55a10930f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- 172.21.15.138:0/1851165790 shutdown_connections 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- 172.21.15.138:0/1851165790 wait complete. 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7b0662880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1092468c0 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7aee07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7aee07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43652/0 (socket says 172.21.15.138:43652) 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7aee07640 1 -- 172.21.15.138:0/1311894990 learned_addr learned my addr 172.21.15.138:0/1311894990 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.392+0000 7fa7aee07640 1 -- 172.21.15.138:0/1311894990 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a10920b6c0 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7aee07640 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55a10a0c2e10 tx=0x55a109318c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7abe01640 1 -- 172.21.15.138:0/1311894990 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921fe40 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a10920b1e0 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7abe01640 1 -- 172.21.15.138:0/1311894990 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a109fe6000 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7abe01640 1 -- 172.21.15.138:0/1311894990 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921f4a0 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a109278ea0 con 0x55a109207000 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7b0662880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7b0662880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a109246c80 con 0x55a109ffec00 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.393+0000 7fa7abe01640 1 -- 172.21.15.138:0/1311894990 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a109230b00 con 0x55a109207000 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7af608640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7af608640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43664/0 (socket says 172.21.15.138:43664) 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7af608640 1 -- 172.21.15.138:0/2300745186 learned_addr learned my addr 172.21.15.138:0/2300745186 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7abe01640 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109fffc00 0x55a1094e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7abe01640 1 -- 172.21.15.138:0/1311894990 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a109219a40 con 0x55a109207000 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7aee07640 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109fffc00 0x55a1094e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7af608640 1 -- 172.21.15.138:0/2300745186 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a109278d00 con 0x55a109ffec00 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7af608640 1 --2- 172.21.15.138:0/2300745186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55a10a1055c0 tx=0x55a109319c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96180103ab77bca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7aee07640 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109fffc00 0x55a1094e9080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55a10a104300 tx=0x55a109319920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7a95fc640 1 -- 172.21.15.138:0/2300745186 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921fe40 con 0x55a109ffec00 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7a95fc640 1 -- 172.21.15.138:0/2300745186 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a109fe6600 con 0x55a109ffec00 2024-04-27T05:21:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.394+0000 7fa7a95fc640 1 -- 172.21.15.138:0/2300745186 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921f4a0 con 0x55a109ffec00 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- 172.21.15.138:0/2300745186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 msgr2=0x55a1094e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 --2- 172.21.15.138:0/2300745186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55a10a1055c0 tx=0x55a109319c20 comp rx=0 tx=0).stop 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- 172.21.15.138:0/2300745186 shutdown_connections 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 --2- 172.21.15.138:0/2300745186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- 172.21.15.138:0/2300745186 >> 172.21.15.138:0/2300745186 conn(0x55a109207400 msgr2=0x55a109311000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- 172.21.15.138:0/2300745186 shutdown_connections 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- 172.21.15.138:0/2300745186 wait complete. 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7b0662880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10922a280 con 0x55a109ffec00 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7af608640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7af608640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43676/0 (socket says 172.21.15.138:43676) 2024-04-27T05:21:36.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 learned_addr learned my addr 172.21.15.138:0/525152324 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.395+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a109278b60 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7af608640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55a10a0c3410 tx=0x55a1093199e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7a7df9640 1 -- 172.21.15.138:0/525152324 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921fe40 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7a7df9640 1 -- 172.21.15.138:0/525152324 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a109fe6780 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a1092789c0 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7a7df9640 1 -- 172.21.15.138:0/525152324 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a10921f4a0 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a109278820 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.396+0000 7fa7a7df9640 1 -- 172.21.15.138:0/525152324 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a109231600 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7a7df9640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109584000 0x55a1094e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7a7df9640 1 -- 172.21.15.138:0/525152324 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a109219c00 con 0x55a109ffec00 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7ae606640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109584000 0x55a1094e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a109fff000 0x55a1094e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109fff400 con 0x55a109fff000 2024-04-27T05:21:36.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7af608640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a109fff000 0x55a1094e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7ae606640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109584000 0x55a1094e9600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55a109313410 tx=0x55a10954c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7af608640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a109fff000 0x55a1094e9b80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 20 realm 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.397+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109fff800 con 0x55a109fff000 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585000 con 0x55a109fff000 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585400 con 0x55a109fff000 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a109585800 0x55a1094ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585c00 con 0x55a109585800 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7aee07640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a109585800 0x55a1094ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.398+0000 7fa7aee07640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a109585800 0x55a1094ea100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7b0662880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7b0662880 20 searching for the correct realm 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585000 con 0x55a109585800 2024-04-27T05:21:36.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7aee07640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a109585000 0x55a1094eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585c00 con 0x55a109585000 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7ae606640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a109585000 0x55a1094eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.399+0000 7fa7ae606640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a109585000 0x55a1094eac00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.400+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.400+0000 7fa7ae606640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a109585c00 0x55a1094eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.400+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.400+0000 7fa7af608640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a109585c00 0x55a1094eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.400+0000 7fa7af608640 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a109585c00 0x55a1094eb180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.401+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585000 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.402+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585000 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.403+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.404+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585000 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.405+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585000 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585800 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.406+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585c00 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109fff000 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109584c00 con 0x55a109585000 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585400 con 0x55a109fff000 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.407+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109fff400 con 0x55a109fff000 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109fff800 con 0x55a109585800 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 zone default found 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 4 Realm: () 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:36.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 zonegroup default 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0000 con 0x55a109585c00 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0400 con 0x55a109585800 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.408+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0800 con 0x55a109585800 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.409+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.409+0000 7fa7b0662880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.409+0000 7fa7b0662880 20 started sync module instance, tier type = 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.409+0000 7fa7b0662880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.409+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0c00 con 0x55a109585000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.410+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.410+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94150134204672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1000 con 0x55a109585000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.410+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1400 con 0x55a109fff000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.411+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94150134204672] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.411+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.411+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94150134205824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1800 con 0x55a109fff000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.411+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1c00 con 0x55a109585800 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.412+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.412+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94150134206976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0400 con 0x55a109585800 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.412+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0800 con 0x55a109585000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.412+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94150134205824] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.413+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.413+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94150134208128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1c00 con 0x55a109585000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.413+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0000 con 0x55a109fff000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.413+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94150134206976] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.414+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.414+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94150134209280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109fff800 con 0x55a109fff000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.414+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109fff400 con 0x55a109585c00 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.414+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94150134208128] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.416+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94150134209280] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.416+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.416+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94150134210432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a109585400 con 0x55a109585c00 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.416+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10a167800 con 0x55a109585c00 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.417+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94150134210432] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.417+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.417+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94150134211584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10a167400 con 0x55a109585c00 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.417+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0c00 con 0x55a109585c00 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94150134211584] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94150134212736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1000 con 0x55a109585c00 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=0 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=1 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=2 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=3 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=4 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=5 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.418+0000 7fa7b0662880 20 add_watcher() i=6 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.419+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94150134212736] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.419+0000 7fa7b0662880 20 add_watcher() i=7 2024-04-27T05:21:36.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.419+0000 7fa7b0662880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.419+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b0800 con 0x55a109fff000 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.419+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.419+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94150134213888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a1095b1400 con 0x55a109fff000 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94150134213888] v29'98 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa795dd5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7b0662880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7b0662880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7b0662880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.420+0000 7fa7b0662880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10922a8c0 con 0x55a109584c00 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7ae606640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7ae606640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43688/0 (socket says 172.21.15.138:43688) 2024-04-27T05:21:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7ae606640 1 -- 172.21.15.138:0/3148811984 learned_addr learned my addr 172.21.15.138:0/3148811984 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7ae606640 1 -- 172.21.15.138:0/3148811984 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a109278680 con 0x55a109584c00 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7ae606640 1 --2- 172.21.15.138:0/3148811984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55a109569620 tx=0x55a10954c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2b117bba65eec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7955d4640 1 -- 172.21.15.138:0/3148811984 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a1095b2580 con 0x55a109584c00 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7955d4640 1 -- 172.21.15.138:0/3148811984 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a109fe6000 con 0x55a109584c00 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.421+0000 7fa7955d4640 1 -- 172.21.15.138:0/3148811984 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a1095b26e0 con 0x55a109584c00 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- 172.21.15.138:0/3148811984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 msgr2=0x55a1094ea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 --2- 172.21.15.138:0/3148811984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x55a109569620 tx=0x55a10954c180 comp rx=0 tx=0).stop 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- 172.21.15.138:0/3148811984 shutdown_connections 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 --2- 172.21.15.138:0/3148811984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- 172.21.15.138:0/3148811984 >> 172.21.15.138:0/3148811984 conn(0x55a1095b1800 msgr2=0x55a109311800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- 172.21.15.138:0/3148811984 shutdown_connections 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- 172.21.15.138:0/3148811984 wait complete. 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 Processor -- start 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- start start 2024-04-27T05:21:36.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.422+0000 7fa7b0662880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10922a780 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7ae606640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7ae606640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43704/0 (socket says 172.21.15.138:43704) 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7ae606640 1 -- 172.21.15.138:0/265058119 learned_addr learned my addr 172.21.15.138:0/265058119 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7ae606640 1 -- 172.21.15.138:0/265058119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1092784e0 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7ae606640 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55a109569830 tx=0x55a10954cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa793dd1640 1 -- 172.21.15.138:0/265058119 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a1095b2840 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa793dd1640 1 -- 172.21.15.138:0/265058119 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a109fe6a80 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a10920b380 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa793dd1640 1 -- 172.21.15.138:0/265058119 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a1095b2580 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.423+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a10920aea0 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa793dd1640 1 -- 172.21.15.138:0/265058119 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a109231b80 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa793dd1640 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a10a167800 0x55a1094eb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa793dd1640 1 -- 172.21.15.138:0/265058119 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55a109219dc0 con 0x55a109584c00 2024-04-27T05:21:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa7b0662880 5 note: GC not initialized 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa7af608640 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a10a167800 0x55a1094eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa7915cc640 20 reqs_thread_entry: start 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa7b0662880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa7af608640 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a10a167800 0x55a1094eb700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55a10a1726c0 tx=0x55a10954c8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa7b0662880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.424+0000 7fa78f5c8640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa78eda6640 20 reqs_thread_entry: start 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa7b0662880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967e000 con 0x55a109fff000 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa7b0662880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967e400 con 0x55a109585000 2024-04-27T05:21:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.425+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.426+0000 7fa795dd5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.426+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94150134213888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967e800 con 0x55a109fff000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.427+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94150134213888] v29'99 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.427+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94150134204672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967ec00 con 0x55a109585000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.428+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94150134204672] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.428+0000 7fa7b0662880 20 remove_watcher() i=0 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.428+0000 7fa7b0662880 2 removed watcher, disabling cache 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.428+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94150134205824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967f000 con 0x55a109fff000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.429+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94150134205824] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.429+0000 7fa7b0662880 20 remove_watcher() i=1 2024-04-27T05:21:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.429+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94150134206976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967f400 con 0x55a109585800 2024-04-27T05:21:36.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.430+0000 7fa7aee07640 1 -- 172.21.15.138:0/525152324 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94150134206976] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959efc0 con 0x55a109585800 2024-04-27T05:21:36.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.430+0000 7fa7b0662880 20 remove_watcher() i=2 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.430+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94150134208128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967f800 con 0x55a109585000 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.431+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94150134208128] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959f200 con 0x55a109585000 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.431+0000 7fa7b0662880 20 remove_watcher() i=3 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.431+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94150134209280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10967fc00 con 0x55a109fff000 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.432+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94150134209280] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109fff000 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.432+0000 7fa7b0662880 20 remove_watcher() i=4 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.432+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94150134210432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10a168000 con 0x55a109585c00 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.432+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94150134210432] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.433+0000 7fa7b0662880 20 remove_watcher() i=5 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.433+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94150134211584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10a168400 con 0x55a109585c00 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.433+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94150134211584] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.433+0000 7fa7b0662880 20 remove_watcher() i=6 2024-04-27T05:21:36.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.433+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94150134212736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a10a168800 con 0x55a109585c00 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7af608640 1 -- 172.21.15.138:0/525152324 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94150134212736] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a10959e900 con 0x55a109585c00 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 20 remove_watcher() i=7 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a109585c00 msgr2=0x55a1094eb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a109585c00 0x55a1094eb180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a109fff000 msgr2=0x55a1094e9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a109fff000 0x55a1094e9b80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a109585000 msgr2=0x55a1094eac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a109585000 0x55a1094eac00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a109585800 msgr2=0x55a1094ea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a109585800 0x55a1094ea100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109584000 msgr2=0x55a1094e9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109584000 0x55a1094e9600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55a109313410 tx=0x55a10954c120 comp rx=0 tx=0).stop 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 msgr2=0x55a1094e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 --2- 172.21.15.138:0/525152324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109ffec00 0x55a1094e8b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55a10a0c3410 tx=0x55a1093199e0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 reap_dead start 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7ae606640 1 -- 172.21.15.138:0/525152324 reap_dead start 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 shutdown_connections 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 >> 172.21.15.138:0/525152324 conn(0x55a109207400 msgr2=0x55a109311000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.434+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 shutdown_connections 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/525152324 wait complete. 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a10a167800 msgr2=0x55a1094eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a10a167800 0x55a1094eb700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55a10a1726c0 tx=0x55a10954c8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 msgr2=0x55a1094ea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55a109569830 tx=0x55a10954cae0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 shutdown_connections 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a10a167800 0x55a1094eb700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/265058119 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109584c00 0x55a1094ea680 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 >> 172.21.15.138:0/265058119 conn(0x55a1095b1800 msgr2=0x55a109311800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 shutdown_connections 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/265058119 wait complete. 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109fffc00 msgr2=0x55a1094e9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109fffc00 0x55a1094e9080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55a10a104300 tx=0x55a109319920 comp rx=0 tx=0).stop 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 msgr2=0x55a1094e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55a10a0c2e10 tx=0x55a109318c60 comp rx=0 tx=0).stop 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 shutdown_connections 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a109fffc00 0x55a1094e9080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 --2- 172.21.15.138:0/1311894990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a109207000 0x55a1094e8580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 >> 172.21.15.138:0/1311894990 conn(0x55a109206c00 msgr2=0x55a10930f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 shutdown_connections 2024-04-27T05:21:36.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.435+0000 7fa7b0662880 1 -- 172.21.15.138:0/1311894990 wait complete. 2024-04-27T05:21:36.452 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-04-27T05:21:36.452 INFO:teuthology.orchestra.run.smithi138.stdout: "myfoo" 2024-04-27T05:21:36.452 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-04-27T05:21:36.452 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-27T05:21:36.456 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.456 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-27T05:21:36.456 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-27T05:21:36.456 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-27T05:21:36.533 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:36.533 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:36.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.531+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f5799a0f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55f5d94d9400 0x55f5d97ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f5799a0f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f5d97b0600 con 0x55f5d94d9000 2024-04-27T05:21:36.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f5799a0f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f5d9518280 con 0x55f5d94d9400 2024-04-27T05:21:36.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f57956db640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55f5d94d9400 0x55f5d97ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f5795edc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55f5d94d9000 0x55f5d95e0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60904/0 (socket says 172.21.15.138:60904) 2024-04-27T05:21:36.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f57956db640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55f5d94d9400 0x55f5d97ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43718/0 (socket says 172.21.15.138:43718) 2024-04-27T05:21:36.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f5795edc640 1 -- 172.21.15.138:0/2332039216 learned_addr learned my addr 172.21.15.138:0/2332039216 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:36.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f57956db640 1 -- 172.21.15.138:0/2332039216 >> v1:172.21.15.114:6789/0 conn(0x55f5d94d9000 legacy=0x55f5d95e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.532+0000 7f57956db640 1 -- 172.21.15.138:0/2332039216 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d94dd380 con 0x55f5d94d9400 2024-04-27T05:21:36.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.534+0000 7f57956db640 1 --2- 172.21.15.138:0/2332039216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9400 0x55f5d97ba580 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f5d95e4c00 tx=0x55f5d95eade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f339db7fea0fbea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.534+0000 7f5794eda640 1 -- 172.21.15.138:0/2332039216 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f1ce0 con 0x55f5d94d9400 2024-04-27T05:21:36.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.534+0000 7f5794eda640 1 -- 172.21.15.138:0/2332039216 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5d97b1b00 con 0x55f5d94d9400 2024-04-27T05:21:36.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.534+0000 7f5794eda640 1 -- 172.21.15.138:0/2332039216 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f1e40 con 0x55f5d94d9400 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- 172.21.15.138:0/2332039216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9400 msgr2=0x55f5d97ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 --2- 172.21.15.138:0/2332039216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9400 0x55f5d97ba580 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55f5d95e4c00 tx=0x55f5d95eade0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- 172.21.15.138:0/2332039216 shutdown_connections 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 --2- 172.21.15.138:0/2332039216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9400 0x55f5d97ba580 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- 172.21.15.138:0/2332039216 >> 172.21.15.138:0/2332039216 conn(0x55f5d94d8c00 msgr2=0x55f5d95e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- 172.21.15.138:0/2332039216 shutdown_connections 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- 172.21.15.138:0/2332039216 wait complete. 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.535+0000 7f5799a0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d9518780 con 0x55f5d94d9000 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f5795edc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f5795edc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43732/0 (socket says 172.21.15.138:43732) 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f5795edc640 1 -- 172.21.15.138:0/3433092293 learned_addr learned my addr 172.21.15.138:0/3433092293 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f5795edc640 1 -- 172.21.15.138:0/3433092293 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d94dcea0 con 0x55f5d94d9000 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f5795edc640 1 --2- 172.21.15.138:0/3433092293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55f5da394cc0 tx=0x55f5d95eacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737b8c639a8f3df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f57946d9640 1 -- 172.21.15.138:0/3433092293 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f14a0 con 0x55f5d94d9000 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f57946d9640 1 -- 172.21.15.138:0/3433092293 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5d97b1e00 con 0x55f5d94d9000 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.536+0000 7f57946d9640 1 -- 172.21.15.138:0/3433092293 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f1ce0 con 0x55f5d94d9000 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- 172.21.15.138:0/3433092293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 msgr2=0x55f5d97ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3433092293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55f5da394cc0 tx=0x55f5d95eacc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- 172.21.15.138:0/3433092293 shutdown_connections 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3433092293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- 172.21.15.138:0/3433092293 >> 172.21.15.138:0/3433092293 conn(0x55f5d94d8c00 msgr2=0x55f5d95e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- 172.21.15.138:0/3433092293 shutdown_connections 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- 172.21.15.138:0/3433092293 wait complete. 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5799a0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d95188c0 con 0x55f5d94d9000 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5795edc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5795edc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43736/0 (socket says 172.21.15.138:43736) 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.537+0000 7f5795edc640 1 -- 172.21.15.138:0/333137276 learned_addr learned my addr 172.21.15.138:0/333137276 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5795edc640 1 -- 172.21.15.138:0/333137276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d94dd6c0 con 0x55f5d94d9000 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5795edc640 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55f5da394ea0 tx=0x55f5d95eac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5792ed6640 1 -- 172.21.15.138:0/333137276 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f1e40 con 0x55f5d94d9000 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5792ed6640 1 -- 172.21.15.138:0/333137276 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5da3aa000 con 0x55f5d94d9000 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5d94dd1e0 con 0x55f5d94d9000 2024-04-27T05:21:36.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5792ed6640 1 -- 172.21.15.138:0/333137276 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f14a0 con 0x55f5d94d9000 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5d954aea0 con 0x55f5d94d9000 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.538+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5799a0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5799a0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d9518c80 con 0x55f5da2d1800 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5792ed6640 1 -- 172.21.15.138:0/333137276 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f5d9503600 con 0x55f5d94d9000 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5792ed6640 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da430000 0x55f5d97bb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57966dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57966dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43752/0 (socket says 172.21.15.138:43752) 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5792ed6640 1 -- 172.21.15.138:0/333137276 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55f5d94eba40 con 0x55f5d94d9000 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57966dd640 1 -- 172.21.15.138:0/405189792 learned_addr learned my addr 172.21.15.138:0/405189792 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5795edc640 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da430000 0x55f5d97bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57966dd640 1 -- 172.21.15.138:0/405189792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d954ad00 con 0x55f5da2d1800 2024-04-27T05:21:36.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57966dd640 1 --2- 172.21.15.138:0/405189792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55f5da395ec0 tx=0x55f5d95eaf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28331643432964e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f5795edc640 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da430000 0x55f5d97bb080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55f5da3de450 tx=0x55f5d95eb4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57906d1640 1 -- 172.21.15.138:0/405189792 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f1e40 con 0x55f5da2d1800 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57906d1640 1 -- 172.21.15.138:0/405189792 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5da3aa300 con 0x55f5da2d1800 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.539+0000 7f57906d1640 1 -- 172.21.15.138:0/405189792 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f14a0 con 0x55f5da2d1800 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- 172.21.15.138:0/405189792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 msgr2=0x55f5d97bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 --2- 172.21.15.138:0/405189792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55f5da395ec0 tx=0x55f5d95eaf00 comp rx=0 tx=0).stop 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- 172.21.15.138:0/405189792 shutdown_connections 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 --2- 172.21.15.138:0/405189792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- 172.21.15.138:0/405189792 >> 172.21.15.138:0/405189792 conn(0x55f5d94d9400 msgr2=0x55f5d95e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- 172.21.15.138:0/405189792 shutdown_connections 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- 172.21.15.138:0/405189792 wait complete. 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f5799a0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d94fc280 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f57966dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f57966dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43754/0 (socket says 172.21.15.138:43754) 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.540+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 learned_addr learned my addr 172.21.15.138:0/3915051354 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d954ab60 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f57966dd640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55f5da2dd470 tx=0x55f5d95ebaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f578eece640 1 -- 172.21.15.138:0/3915051354 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f1e40 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f578eece640 1 -- 172.21.15.138:0/3915051354 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5da3aa480 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5d954a9c0 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f578eece640 1 -- 172.21.15.138:0/3915051354 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5d94f14a0 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.541+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5d954a820 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f578eece640 1 -- 172.21.15.138:0/3915051354 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f5d95038c0 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f578eece640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5d985c000 0x55f5d97bb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f578eece640 1 -- 172.21.15.138:0/3915051354 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55f5d94ebc00 con 0x55f5da2d1800 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f57956db640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5d985c000 0x55f5d97bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f5da2d1c00 0x55f5d97bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985c400 con 0x55f5da2d1c00 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f57966dd640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f5da2d1c00 0x55f5d97bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f57956db640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5d985c000 0x55f5d97bb600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55f5d95e4c00 tx=0x55f5d9826120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.542+0000 7f57966dd640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f5da2d1c00 0x55f5d97bbb80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 realm 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985c800 con 0x55f5da2d1c00 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:36.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985cc00 con 0x55f5da2d1c00 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985d000 con 0x55f5da2d1c00 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f5d985d400 0x55f5d97bc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985c400 con 0x55f5d985d400 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.543+0000 7f5795edc640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f5d985d400 0x55f5d97bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5795edc640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f5d985d400 0x55f5d97bc100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5799a0f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:36.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5799a0f880 20 searching for the correct realm 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985c800 con 0x55f5d985d400 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5795edc640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f5d985c800 0x55f5d97bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985c400 con 0x55f5d985c800 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.544+0000 7f57956db640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f5d985c800 0x55f5d97bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.545+0000 7f57956db640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f5d985c800 0x55f5d97bcc00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.545+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.545+0000 7f57956db640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f5d985c400 0x55f5d97bd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.545+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.545+0000 7f57966dd640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f5d985c400 0x55f5d97bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.545+0000 7f57966dd640 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f5d985c400 0x55f5d97bd180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.546+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c800 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.547+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c800 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.548+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.548+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.548+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.548+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.548+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.548+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.549+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c800 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.550+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c800 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.551+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985d400 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c400 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5da2d1c00 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2d0800 con 0x55f5d985c800 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5799a0f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5799a0f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5799a0f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5799a0f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.552+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985cc00 con 0x55f5da2d1c00 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985d000 con 0x55f5da2d1c00 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985d800 con 0x55f5d985d400 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 zone default found 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 4 Realm: () 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:36.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 zonegroup default 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.553+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985dc00 con 0x55f5d985c400 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc000 con 0x55f5d985d400 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc400 con 0x55f5d985d400 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 started sync module instance, tier type = 2024-04-27T05:21:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.554+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc800 con 0x55f5d985c800 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.555+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.555+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94514404075776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bcc00 con 0x55f5d985c800 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.555+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bd000 con 0x55f5da2d1c00 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.556+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94514404075776] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.556+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.556+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94514404076928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bd400 con 0x55f5da2d1c00 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.556+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bd800 con 0x55f5d985d400 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.557+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94514404076928] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.557+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.557+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94514404078080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc000 con 0x55f5d985d400 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.557+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc400 con 0x55f5d985c800 2024-04-27T05:21:36.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.558+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.558+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94514404079232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bd800 con 0x55f5d985c800 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.558+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94514404078080] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.558+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985d800 con 0x55f5da2d1c00 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.559+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.559+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94514404079232] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.559+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94514404080384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985cc00 con 0x55f5da2d1c00 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.559+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985d000 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.560+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.560+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94514404081536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d985dc00 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.560+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94514404080384] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.560+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da431800 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.561+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.561+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94514404082688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc800 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.561+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bcc00 con 0x55f5d985c400 2024-04-27T05:21:36.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.561+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94514404081536] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94514404083840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bc400 con 0x55f5d985c400 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=0 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=1 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=2 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=3 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=4 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=5 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94514404082688] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.562+0000 7f5799a0f880 20 add_watcher() i=6 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.563+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94514404083840] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.563+0000 7f5799a0f880 20 add_watcher() i=7 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.563+0000 7f5799a0f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.563+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bd800 con 0x55f5da2d1c00 2024-04-27T05:21:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.563+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.563+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94514404084992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5da2bd000 con 0x55f5da2d1c00 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.564+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94514404084992] v29'100 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.564+0000 7f577ceaa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.564+0000 7f5799a0f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.564+0000 7f5799a0f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.564+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f5799a0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f5799a0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d94fc8c0 con 0x55f5da2bdc00 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f57956db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f57956db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43770/0 (socket says 172.21.15.138:43770) 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f57956db640 1 -- 172.21.15.138:0/1560898072 learned_addr learned my addr 172.21.15.138:0/1560898072 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f57956db640 1 -- 172.21.15.138:0/1560898072 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d954a680 con 0x55f5da2bdc00 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.565+0000 7f57956db640 1 --2- 172.21.15.138:0/1560898072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55f5da450780 tx=0x55f5d98260c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be85ddbc932ca041 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f577c6a9640 1 -- 172.21.15.138:0/1560898072 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5da2be580 con 0x55f5da2bdc00 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f577c6a9640 1 -- 172.21.15.138:0/1560898072 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5da3aa000 con 0x55f5da2bdc00 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f577c6a9640 1 -- 172.21.15.138:0/1560898072 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5da2be6e0 con 0x55f5da2bdc00 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 -- 172.21.15.138:0/1560898072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 msgr2=0x55f5d97bc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 --2- 172.21.15.138:0/1560898072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55f5da450780 tx=0x55f5d98260c0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 -- 172.21.15.138:0/1560898072 shutdown_connections 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 --2- 172.21.15.138:0/1560898072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 -- 172.21.15.138:0/1560898072 >> 172.21.15.138:0/1560898072 conn(0x55f5da2bd400 msgr2=0x55f5d95e3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 -- 172.21.15.138:0/1560898072 shutdown_connections 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 -- 172.21.15.138:0/1560898072 wait complete. 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.566+0000 7f5799a0f880 1 Processor -- start 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f5799a0f880 1 -- start start 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f5799a0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f5799a0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d94fc780 con 0x55f5da2bdc00 2024-04-27T05:21:36.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f57956db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f57956db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43774/0 (socket says 172.21.15.138:43774) 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f57956db640 1 -- 172.21.15.138:0/2209098523 learned_addr learned my addr 172.21.15.138:0/2209098523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f57956db640 1 -- 172.21.15.138:0/2209098523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d954a4e0 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f57956db640 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55f5d983f740 tx=0x55f5d9826000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.567+0000 7f577aea6640 1 -- 172.21.15.138:0/2209098523 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5da2be840 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f577aea6640 1 -- 172.21.15.138:0/2209098523 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5da3aa780 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5d94dcea0 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f577aea6640 1 -- 172.21.15.138:0/2209098523 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f5da2be580 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5d94dd6c0 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f577aea6640 1 -- 172.21.15.138:0/2209098523 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55f5d9502b00 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f577aea6640 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da2bc800 0x55f5d97bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f577aea6640 1 -- 172.21.15.138:0/2209098523 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55f5d94ebdc0 con 0x55f5da2bdc00 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f5799a0f880 5 note: GC not initialized 2024-04-27T05:21:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f57966dd640 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da2bc800 0x55f5d97bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.568+0000 7f57786a1640 20 reqs_thread_entry: start 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f5799a0f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f57966dd640 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da2bc800 0x55f5d97bd700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55f5da450bd0 tx=0x55f5d9826960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f5799a0f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f577669d640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:06 2024 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f5775e7b640 20 reqs_thread_entry: start 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(62 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.569+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(63 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(64 [pgnls start_epoch 29 out=49b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 [pgnls start_epoch 29 out=49b] v29'11 uv11 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c800 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(67 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.570+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(69 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c800 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(70 [pgnls start_epoch 29 out=127b] v29'3 uv3 ondisk = 1) v8 ==== 144+0+127 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(71 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.571+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(73 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(74 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(75 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.572+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(76 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(77 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(78 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(79 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c800 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(80 [pgnls start_epoch 29 out=70b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+70 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.573+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(81 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(82 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(83 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c800 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(84 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.574+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(85 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(86 [pgnls start_epoch 29 out=49b] v29'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(87 [pgnls start_epoch 29 out=49b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(88 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.575+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c800 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(89 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(90 [pgnls start_epoch 29 out=49b] v29'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5da2d1c00 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(91 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985d400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(92 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.576+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c000 con 0x55f5d985c800 2024-04-27T05:21:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.577+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(93 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.577+0000 7f5799a0f880 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.577+0000 7f5799a0f880 20 RGWRados::pool_iterate: got myfoo 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.577+0000 7f577ceaa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.577+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94514404084992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c400 con 0x55f5da2d1c00 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.578+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94514404084992] v29'101 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.578+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94514404075776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994c800 con 0x55f5d985c800 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.579+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94514404075776] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.579+0000 7f5799a0f880 20 remove_watcher() i=0 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.579+0000 7f5799a0f880 2 removed watcher, disabling cache 2024-04-27T05:21:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.579+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94514404076928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994cc00 con 0x55f5da2d1c00 2024-04-27T05:21:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.580+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94514404076928] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.580+0000 7f5799a0f880 20 remove_watcher() i=1 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.580+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94514404078080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994d000 con 0x55f5d985d400 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.581+0000 7f5795edc640 1 -- 172.21.15.138:0/3915051354 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94514404078080] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986ed80 con 0x55f5d985d400 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.581+0000 7f5799a0f880 20 remove_watcher() i=2 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.581+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94514404079232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994d400 con 0x55f5d985c800 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.582+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94514404079232] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986efc0 con 0x55f5d985c800 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.582+0000 7f5799a0f880 20 remove_watcher() i=3 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.582+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94514404080384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994d800 con 0x55f5da2d1c00 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.583+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94514404080384] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5da2d1c00 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.583+0000 7f5799a0f880 20 remove_watcher() i=4 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.583+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94514404081536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d994dc00 con 0x55f5d985c400 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.584+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94514404081536] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.584+0000 7f5799a0f880 20 remove_watcher() i=5 2024-04-27T05:21:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.584+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94514404082688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d986c000 con 0x55f5d985c400 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.585+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94514404082688] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.585+0000 7f5799a0f880 20 remove_watcher() i=6 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.585+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94514404083840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5d986c400 con 0x55f5d985c400 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f57966dd640 1 -- 172.21.15.138:0/3915051354 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94514404083840] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f5d986e6c0 con 0x55f5d985c400 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 20 remove_watcher() i=7 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f5d985c400 msgr2=0x55f5d97bd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55f5d985c400 0x55f5d97bd180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f5da2d1c00 msgr2=0x55f5d97bbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55f5da2d1c00 0x55f5d97bbb80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f5d985c800 msgr2=0x55f5d97bcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55f5d985c800 0x55f5d97bcc00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f5d985d400 msgr2=0x55f5d97bc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55f5d985d400 0x55f5d97bc100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5d985c000 msgr2=0x55f5d97bb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5d985c000 0x55f5d97bb600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55f5d95e4c00 tx=0x55f5d9826120 comp rx=0 tx=0).stop 2024-04-27T05:21:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 msgr2=0x55f5d97bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 --2- 172.21.15.138:0/3915051354 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2d1800 0x55f5d97bab00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55f5da2dd470 tx=0x55f5d95ebaa0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 reap_dead start 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f57956db640 1 -- 172.21.15.138:0/3915051354 reap_dead start 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 shutdown_connections 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 >> 172.21.15.138:0/3915051354 conn(0x55f5d94d9400 msgr2=0x55f5d95e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 shutdown_connections 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.586+0000 7f5799a0f880 1 -- 172.21.15.138:0/3915051354 wait complete. 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da2bc800 msgr2=0x55f5d97bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da2bc800 0x55f5d97bd700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55f5da450bd0 tx=0x55f5d9826960 comp rx=0 tx=0).stop 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 msgr2=0x55f5d97bc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55f5d983f740 tx=0x55f5d9826000 comp rx=0 tx=0).stop 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 shutdown_connections 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da2bc800 0x55f5d97bd700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/2209098523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5da2bdc00 0x55f5d97bc680 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 >> 172.21.15.138:0/2209098523 conn(0x55f5da2bd400 msgr2=0x55f5d95e3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 shutdown_connections 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/2209098523 wait complete. 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da430000 msgr2=0x55f5d97bb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da430000 0x55f5d97bb080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55f5da3de450 tx=0x55f5d95eb4a0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 msgr2=0x55f5d97ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55f5da394ea0 tx=0x55f5d95eac60 comp rx=0 tx=0).stop 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 shutdown_connections 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55f5da430000 0x55f5d97bb080 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 --2- 172.21.15.138:0/333137276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f5d94d9000 0x55f5d97ba580 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 >> 172.21.15.138:0/333137276 conn(0x55f5d94d8c00 msgr2=0x55f5d95e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 shutdown_connections 2024-04-27T05:21:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.587+0000 7f5799a0f880 1 -- 172.21.15.138:0/333137276 wait complete. 2024-04-27T05:21:36.606 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-04-27T05:21:36.606 INFO:teuthology.orchestra.run.smithi138.stdout: "myfoo" 2024-04-27T05:21:36.606 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-04-27T05:21:36.606 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-27T05:21:36.616 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.629 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.641 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.689 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-04-27T05:21:36.720 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.748 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.776 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:36.776 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T05:21:36.776 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T05:21:36.776 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T05:21:36.818 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:36.819 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:36.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff661b31880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x564533531400 0x564533812580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff661b31880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564533808600 con 0x564533531000 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff661b31880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x564533570280 con 0x564533531400 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff65dd6e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x564533531400 0x564533812580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff65dd6e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x564533531400 0x564533812580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43790/0 (socket says 172.21.15.138:43790) 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff65e56f640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x564533531000 0x564533638800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60914/0 (socket says 172.21.15.138:60914) 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff65dd6e640 1 -- 172.21.15.138:0/1654753614 learned_addr learned my addr 172.21.15.138:0/1654753614 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff65dd6e640 1 -- 172.21.15.138:0/1654753614 >> v1:172.21.15.114:6789/0 conn(0x564533531000 legacy=0x564533638800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.817+0000 7ff65dd6e640 1 -- 172.21.15.138:0/1654753614 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564533535380 con 0x564533531400 2024-04-27T05:21:36.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.819+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/1654753614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531400 0x564533812580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x56453363cc00 tx=0x564533642de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ea17ffa1f606c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.819+0000 7ff65d56d640 1 -- 172.21.15.138:0/1654753614 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564533549ce0 con 0x564533531400 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.819+0000 7ff65d56d640 1 -- 172.21.15.138:0/1654753614 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564533809b00 con 0x564533531400 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.819+0000 7ff65d56d640 1 -- 172.21.15.138:0/1654753614 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564533549e40 con 0x564533531400 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 -- 172.21.15.138:0/1654753614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531400 msgr2=0x564533812580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 --2- 172.21.15.138:0/1654753614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531400 0x564533812580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x56453363cc00 tx=0x564533642de0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 -- 172.21.15.138:0/1654753614 shutdown_connections 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 --2- 172.21.15.138:0/1654753614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531400 0x564533812580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 -- 172.21.15.138:0/1654753614 >> 172.21.15.138:0/1654753614 conn(0x564533530c00 msgr2=0x564533638000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 -- 172.21.15.138:0/1654753614 shutdown_connections 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 -- 172.21.15.138:0/1654753614 wait complete. 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.820+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff661b31880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff661b31880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564533570780 con 0x564533531000 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65e56f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65e56f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43798/0 (socket says 172.21.15.138:43798) 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65e56f640 1 -- 172.21.15.138:0/2322269786 learned_addr learned my addr 172.21.15.138:0/2322269786 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65e56f640 1 -- 172.21.15.138:0/2322269786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564533534ea0 con 0x564533531000 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65e56f640 1 --2- 172.21.15.138:0/2322269786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5645343ed290 tx=0x564533642cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fda0841a77bf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65cd6c640 1 -- 172.21.15.138:0/2322269786 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645335494a0 con 0x564533531000 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65cd6c640 1 -- 172.21.15.138:0/2322269786 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564533809e00 con 0x564533531000 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.821+0000 7ff65cd6c640 1 -- 172.21.15.138:0/2322269786 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564533549ce0 con 0x564533531000 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- 172.21.15.138:0/2322269786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 msgr2=0x564533812580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 --2- 172.21.15.138:0/2322269786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5645343ed290 tx=0x564533642cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- 172.21.15.138:0/2322269786 shutdown_connections 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 --2- 172.21.15.138:0/2322269786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- 172.21.15.138:0/2322269786 >> 172.21.15.138:0/2322269786 conn(0x564533530c00 msgr2=0x564533639000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- 172.21.15.138:0/2322269786 shutdown_connections 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- 172.21.15.138:0/2322269786 wait complete. 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff661b31880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645335708c0 con 0x564533531000 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff65e56f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff65e56f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43810/0 (socket says 172.21.15.138:43810) 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.822+0000 7ff65e56f640 1 -- 172.21.15.138:0/1781291055 learned_addr learned my addr 172.21.15.138:0/1781291055 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff65e56f640 1 -- 172.21.15.138:0/1781291055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645335356c0 con 0x564533531000 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff65e56f640 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5645343ecb40 tx=0x564533642c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff65b569640 1 -- 172.21.15.138:0/1781291055 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564533549e40 con 0x564533531000 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff65b569640 1 -- 172.21.15.138:0/1781291055 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564534402000 con 0x564533531000 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645335351e0 con 0x564533531000 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff65b569640 1 -- 172.21.15.138:0/1781291055 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645335494a0 con 0x564533531000 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645335a2ea0 con 0x564533531000 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.823+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff661b31880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff661b31880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564533570c80 con 0x564534329800 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65b569640 1 -- 172.21.15.138:0/1781291055 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56453355b600 con 0x564533531000 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65ed70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65b569640 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56453448c000 0x564533813080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65ed70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43816/0 (socket says 172.21.15.138:43816) 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65ed70640 1 -- 172.21.15.138:0/54105875 learned_addr learned my addr 172.21.15.138:0/54105875 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65b569640 1 -- 172.21.15.138:0/1781291055 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x564533543a40 con 0x564533531000 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65e56f640 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56453448c000 0x564533813080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65ed70640 1 -- 172.21.15.138:0/54105875 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645335a2d00 con 0x564534329800 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65ed70640 1 --2- 172.21.15.138:0/54105875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5645343edd10 tx=0x564533642f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66394f57e02c66e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff65e56f640 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56453448c000 0x564533813080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x564534434510 tx=0x5645336434a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff658d64640 1 -- 172.21.15.138:0/54105875 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564533549e40 con 0x564534329800 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff658d64640 1 -- 172.21.15.138:0/54105875 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564534402300 con 0x564534329800 2024-04-27T05:21:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.824+0000 7ff658d64640 1 -- 172.21.15.138:0/54105875 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645335494a0 con 0x564534329800 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- 172.21.15.138:0/54105875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 msgr2=0x564533812b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 --2- 172.21.15.138:0/54105875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5645343edd10 tx=0x564533642f00 comp rx=0 tx=0).stop 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- 172.21.15.138:0/54105875 shutdown_connections 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 --2- 172.21.15.138:0/54105875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- 172.21.15.138:0/54105875 >> 172.21.15.138:0/54105875 conn(0x564533531400 msgr2=0x56453363b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- 172.21.15.138:0/54105875 shutdown_connections 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- 172.21.15.138:0/54105875 wait complete. 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff661b31880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564533554280 con 0x564534329800 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff65ed70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff65ed70640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43832/0 (socket says 172.21.15.138:43832) 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.825+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 learned_addr learned my addr 172.21.15.138:0/3389560833 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645335a2b60 con 0x564534329800 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff65ed70640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5645343ed500 tx=0x564533643aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff657561640 1 -- 172.21.15.138:0/3389560833 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564533549e40 con 0x564534329800 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff657561640 1 -- 172.21.15.138:0/3389560833 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564534402480 con 0x564534329800 2024-04-27T05:21:36.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645335a29c0 con 0x564534329800 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff657561640 1 -- 172.21.15.138:0/3389560833 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645335494a0 con 0x564534329800 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.826+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645335a2820 con 0x564534329800 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff657561640 1 -- 172.21.15.138:0/3389560833 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56453355bb80 con 0x564534329800 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff657561640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5645338ba000 0x564533813600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff657561640 1 -- 172.21.15.138:0/3389560833 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x564533543c00 con 0x564534329800 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5645338ba000 0x564533813600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564534329c00 0x564533813b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338ba400 con 0x564534329c00 2024-04-27T05:21:36.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff65ed70640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564534329c00 0x564533813b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5645338ba000 0x564533813600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x56453363cc00 tx=0x56453387c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.827+0000 7ff65ed70640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564534329c00 0x564533813b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 realm 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338ba800 con 0x564534329c00 2024-04-27T05:21:36.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bac00 con 0x564534329c00 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bb000 con 0x564534329c00 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5645338bb400 0x564533814100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.828+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338ba400 con 0x5645338bb400 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff65e56f640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5645338bb400 0x564533814100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff65e56f640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5645338bb400 0x564533814100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:36.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff661b31880 20 searching for the correct realm 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338ba800 con 0x5645338bb400 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff65e56f640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5645338ba800 0x564533814c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.829+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338ba400 con 0x5645338ba800 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.830+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5645338ba800 0x564533814c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.830+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5645338ba800 0x564533814c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.830+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.830+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5645338ba400 0x564533815180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.830+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.830+0000 7ff65ed70640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5645338ba400 0x564533815180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5645338ba400 0x564533815180 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.831+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba800 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.832+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba800 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.833+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.834+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.834+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.834+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.834+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.834+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.834+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.835+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba800 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.836+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba800 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338bb400 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba400 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.837+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x564534329c00 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534328800 con 0x5645338ba800 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:36.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bac00 con 0x564534329c00 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bb000 con 0x564534329c00 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:36.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bb800 con 0x5645338bb400 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.838+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 zone default found 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 4 Realm: () 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 zonegroup default 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bbc00 con 0x5645338ba400 2024-04-27T05:21:36.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314000 con 0x5645338bb400 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314400 con 0x5645338bb400 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 started sync module instance, tier type = 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.839+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314800 con 0x5645338ba800 2024-04-27T05:21:36.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.840+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.840+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94855216802048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314c00 con 0x5645338ba800 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.840+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534315000 con 0x564534329c00 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.841+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94855216802048] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.841+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.841+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94855216803200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534315400 con 0x564534329c00 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.841+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534315800 con 0x5645338bb400 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.842+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94855216803200] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.843+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.843+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94855216804352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314000 con 0x5645338bb400 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.843+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314400 con 0x5645338ba800 2024-04-27T05:21:36.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.843+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94855216804352] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.844+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.844+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94855216805504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534315800 con 0x5645338ba800 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.844+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bb800 con 0x564534329c00 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.845+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.845+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94855216806656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bac00 con 0x564534329c00 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.845+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bb000 con 0x5645338ba400 2024-04-27T05:21:36.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.845+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94855216805504] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.845+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.846+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94855216807808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bbc00 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.846+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56453448d800 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.846+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94855216806656] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.846+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94855216807808] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.847+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.847+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94855216808960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314800 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.847+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314c00 con 0x5645338ba400 2024-04-27T05:21:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94855216808960] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94855216810112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314400 con 0x5645338ba400 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=0 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=1 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=2 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=3 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=4 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=5 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=6 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94855216810112] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 20 add_watcher() i=7 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.848+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534315800 con 0x564534329c00 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.849+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.849+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94855216811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534315000 con 0x564534329c00 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94855216811264] v29'102 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff64553d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff661b31880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff661b31880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff661b31880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff661b31880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645335548c0 con 0x564534315c00 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff65dd6e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff65dd6e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43846/0 (socket says 172.21.15.138:43846) 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.850+0000 7ff65dd6e640 1 -- 172.21.15.138:0/2258753911 learned_addr learned my addr 172.21.15.138:0/2258753911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.851+0000 7ff65dd6e640 1 -- 172.21.15.138:0/2258753911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645335a2680 con 0x564534315c00 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.851+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/2258753911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x564533896b70 tx=0x56453387c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2748b1a18e23df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.851+0000 7ff644d3c640 1 -- 172.21.15.138:0/2258753911 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564534316580 con 0x564534315c00 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.851+0000 7ff644d3c640 1 -- 172.21.15.138:0/2258753911 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564534402000 con 0x564534315c00 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.851+0000 7ff644d3c640 1 -- 172.21.15.138:0/2258753911 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5645343166e0 con 0x564534315c00 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- 172.21.15.138:0/2258753911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 msgr2=0x564533814680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 --2- 172.21.15.138:0/2258753911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x564533896b70 tx=0x56453387c0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- 172.21.15.138:0/2258753911 shutdown_connections 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 --2- 172.21.15.138:0/2258753911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- 172.21.15.138:0/2258753911 >> 172.21.15.138:0/2258753911 conn(0x564534315400 msgr2=0x56453363b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- 172.21.15.138:0/2258753911 shutdown_connections 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- 172.21.15.138:0/2258753911 wait complete. 2024-04-27T05:21:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 Processor -- start 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- start start 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff661b31880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564533554780 con 0x564534315c00 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff65dd6e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff65dd6e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43852/0 (socket says 172.21.15.138:43852) 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.852+0000 7ff65dd6e640 1 -- 172.21.15.138:0/743816560 learned_addr learned my addr 172.21.15.138:0/743816560 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff65dd6e640 1 -- 172.21.15.138:0/743816560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645335a24e0 con 0x564534315c00 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff65dd6e640 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x564533897560 tx=0x56453387c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff643539640 1 -- 172.21.15.138:0/743816560 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564534316840 con 0x564534315c00 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff643539640 1 -- 172.21.15.138:0/743816560 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564534402780 con 0x564534315c00 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564533534ea0 con 0x564534315c00 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff643539640 1 -- 172.21.15.138:0/743816560 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564534316580 con 0x564534315c00 2024-04-27T05:21:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.853+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645335356c0 con 0x564534315c00 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff643539640 1 -- 172.21.15.138:0/743816560 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56453355ab00 con 0x564534315c00 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff643539640 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564534314800 0x564533815700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff643539640 1 -- 172.21.15.138:0/743816560 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x564533543dc0 con 0x564534315c00 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff661b31880 5 note: GC not initialized 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff65ed70640 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564534314800 0x564533815700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff640d34640 20 reqs_thread_entry: start 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff661b31880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff65ed70640 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564534314800 0x564533815700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5645344a69c0 tx=0x56453387c960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff661b31880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff63ed30640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:07 2024 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff63e50e640 20 reqs_thread_entry: start 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.854+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4000 con 0x5645338ba800 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4400 con 0x5645338ba800 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.855+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4800 con 0x5645338ba800 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 20 rados->read ofs=0 len=0 2024-04-27T05:21:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4c00 con 0x5645338ba800 2024-04-27T05:21:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(65 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:36.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:36.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 4.34 4:2e3daf19:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5000 con 0x5645338ba800 2024-04-27T05:21:36.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.35 4:adc4a58e:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5400 con 0x564534329c00 2024-04-27T05:21:36.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 4.2c 4:346a3ea0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5800 con 0x564534329c00 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 4.2e 4:74cdb6e4:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5c00 con 0x5645338bb400 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 4.3a 4:5decd619:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4000 con 0x5645338bb400 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 4.c 4:335b21e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4400 con 0x564534329c00 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4800 con 0x564534329c00 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.2f 4:f6a773d6:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4c00 con 0x564534329c00 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.2 4:4123e525:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5000 con 0x564534329c00 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 4.3e 4:7d356189:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bbc00 con 0x5645338ba800 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.856+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 4.16 4:6823909f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645338bb000 con 0x5645338bb400 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.857+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(76 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.858+0000 7ff64553d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.858+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94855216811264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5400 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.859+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94855216811264] v29'103 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.859+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94855216802048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a5800 con 0x5645338ba800 2024-04-27T05:21:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.860+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94855216802048] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.860+0000 7ff661b31880 20 remove_watcher() i=0 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.860+0000 7ff661b31880 2 removed watcher, disabling cache 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.860+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94855216803200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4400 con 0x564534329c00 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.861+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94855216803200] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.861+0000 7ff661b31880 20 remove_watcher() i=1 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.861+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94855216804352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4800 con 0x5645338bb400 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.862+0000 7ff65e56f640 1 -- 172.21.15.138:0/3389560833 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94855216804352] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cafc0 con 0x5645338bb400 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.862+0000 7ff661b31880 20 remove_watcher() i=2 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.862+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94855216805504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5645339a4c00 con 0x5645338ba800 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.863+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94855216805504] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338cb200 con 0x5645338ba800 2024-04-27T05:21:36.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.863+0000 7ff661b31880 20 remove_watcher() i=3 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.863+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94855216806656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314c00 con 0x564534329c00 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.864+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94855216806656] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x564534329c00 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.864+0000 7ff661b31880 20 remove_watcher() i=4 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.864+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94855216807808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x564534314400 con 0x5645338ba400 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.865+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94855216807808] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.865+0000 7ff661b31880 20 remove_watcher() i=5 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.865+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94855216808960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56453385c000 con 0x5645338ba400 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.865+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94855216808960] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.865+0000 7ff661b31880 20 remove_watcher() i=6 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.865+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94855216810112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56453385c400 con 0x5645338ba400 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff65ed70640 1 -- 172.21.15.138:0/3389560833 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94855216810112] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5645338ca900 con 0x5645338ba400 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 20 remove_watcher() i=7 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5645338ba400 msgr2=0x564533815180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5645338ba400 0x564533815180 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564534329c00 msgr2=0x564533813b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564534329c00 0x564533813b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5645338ba800 msgr2=0x564533814c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5645338ba800 0x564533814c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5645338bb400 msgr2=0x564533814100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5645338bb400 0x564533814100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5645338ba000 msgr2=0x564533813600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5645338ba000 0x564533813600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x56453363cc00 tx=0x56453387c120 comp rx=0 tx=0).stop 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 msgr2=0x564533812b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/3389560833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534329800 0x564533812b00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5645343ed500 tx=0x564533643aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 reap_dead start 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff65dd6e640 1 -- 172.21.15.138:0/3389560833 reap_dead start 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 shutdown_connections 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 >> 172.21.15.138:0/3389560833 conn(0x564533531400 msgr2=0x56453363b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 shutdown_connections 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/3389560833 wait complete. 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564534314800 msgr2=0x564533815700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564534314800 0x564533815700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5645344a69c0 tx=0x56453387c960 comp rx=0 tx=0).stop 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 msgr2=0x564533814680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x564533897560 tx=0x56453387c000 comp rx=0 tx=0).stop 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 shutdown_connections 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564534314800 0x564533815700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 --2- 172.21.15.138:0/743816560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564534315c00 0x564533814680 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 >> 172.21.15.138:0/743816560 conn(0x564534315400 msgr2=0x56453363b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 shutdown_connections 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.867+0000 7ff661b31880 1 -- 172.21.15.138:0/743816560 wait complete. 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56453448c000 msgr2=0x564533813080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56453448c000 0x564533813080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x564534434510 tx=0x5645336434a0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 msgr2=0x564533812580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5645343ecb40 tx=0x564533642c60 comp rx=0 tx=0).stop 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 shutdown_connections 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56453448c000 0x564533813080 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 --2- 172.21.15.138:0/1781291055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564533531000 0x564533812580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 >> 172.21.15.138:0/1781291055 conn(0x564533530c00 msgr2=0x564533639000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 shutdown_connections 2024-04-27T05:21:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.868+0000 7ff661b31880 1 -- 172.21.15.138:0/1781291055 wait complete. 2024-04-27T05:21:36.901 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:36.901 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:21:36.901 INFO:teuthology.orchestra.run.smithi138.stdout: "num_shards": 11, 2024-04-27T05:21:36.901 INFO:teuthology.orchestra.run.smithi138.stdout: "tenant": "", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "zonegroup": "7c1a7efe-08a0-41e8-a0e4-e18cf496a33b", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_rule": "default-placement", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "explicit_placement": { 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "" 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "marker": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": "Normal", 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "versioned": false, 2024-04-27T05:21:36.902 INFO:teuthology.orchestra.run.smithi138.stdout: "versioning_enabled": false, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "object_lock_enabled": false, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_enabled": false, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "mtime": "2024-04-27T05:21:36.332917Z", 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "creation_time": "2024-04-27T05:21:36.328858Z", 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "usage": {}, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:36.903 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:36.904 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:36.904 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:36.904 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:21:36.332917Z', 'creation_time': '2024-04-27T05:21:36.328858Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T05:21:36.904 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-27T05:21:36.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-27T05:21:36.904 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-27T05:21:36.983 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:36.984 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.981+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3e0c86880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55df7e2c3400 0x55df7e5a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3e0c86880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55df7e59a600 con 0x55df7e2c3000 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3e0c86880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55df7e302280 con 0x55df7e2c3400 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3dec2a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55df7e2c3400 0x55df7e5a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3df42b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55df7e2c3000 0x55df7e3ca800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60926/0 (socket says 172.21.15.138:60926) 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3df42b640 1 -- 172.21.15.138:0/1578558706 learned_addr learned my addr 172.21.15.138:0/1578558706 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3dec2a640 1 -- 172.21.15.138:0/1578558706 >> v1:172.21.15.114:6789/0 conn(0x55df7e2c3000 legacy=0x55df7e3ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.982+0000 7ff3dec2a640 1 -- 172.21.15.138:0/1578558706 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e2c7380 con 0x55df7e2c3400 2024-04-27T05:21:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.984+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/1578558706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3400 0x55df7e5a4580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55df7e3cec00 tx=0x55df7e3d4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd58f19f3f5d4605 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.984+0000 7ff3de429640 1 -- 172.21.15.138:0/1578558706 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2dbce0 con 0x55df7e2c3400 2024-04-27T05:21:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.984+0000 7ff3de429640 1 -- 172.21.15.138:0/1578558706 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7e59bb00 con 0x55df7e2c3400 2024-04-27T05:21:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.984+0000 7ff3de429640 1 -- 172.21.15.138:0/1578558706 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2dbe40 con 0x55df7e2c3400 2024-04-27T05:21:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.984+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1578558706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3400 msgr2=0x55df7e5a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.984+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/1578558706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3400 0x55df7e5a4580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55df7e3cec00 tx=0x55df7e3d4de0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1578558706 shutdown_connections 2024-04-27T05:21:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/1578558706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3400 0x55df7e5a4580 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1578558706 >> 172.21.15.138:0/1578558706 conn(0x55df7e2c2c00 msgr2=0x55df7e3ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1578558706 shutdown_connections 2024-04-27T05:21:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1578558706 wait complete. 2024-04-27T05:21:37.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:37.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:37.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.985+0000 7ff3e0c86880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7e302780 con 0x55df7e2c3000 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3df42b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3df42b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43858/0 (socket says 172.21.15.138:43858) 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3df42b640 1 -- 172.21.15.138:0/2348581536 learned_addr learned my addr 172.21.15.138:0/2348581536 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3df42b640 1 -- 172.21.15.138:0/2348581536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e2c6ea0 con 0x55df7e2c3000 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3df42b640 1 --2- 172.21.15.138:0/2348581536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55df7f17ecc0 tx=0x55df7e3d4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275dcb3fe01eea88 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3ddc28640 1 -- 172.21.15.138:0/2348581536 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2db4a0 con 0x55df7e2c3000 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3ddc28640 1 -- 172.21.15.138:0/2348581536 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7e59be00 con 0x55df7e2c3000 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3ddc28640 1 -- 172.21.15.138:0/2348581536 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2dbce0 con 0x55df7e2c3000 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2348581536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 msgr2=0x55df7e5a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.986+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/2348581536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55df7f17ecc0 tx=0x55df7e3d4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2348581536 shutdown_connections 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/2348581536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2348581536 >> 172.21.15.138:0/2348581536 conn(0x55df7e2c2c00 msgr2=0x55df7e3cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2348581536 shutdown_connections 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2348581536 wait complete. 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3e0c86880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7e3028c0 con 0x55df7e2c3000 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3df42b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3df42b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43864/0 (socket says 172.21.15.138:43864) 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3df42b640 1 -- 172.21.15.138:0/1930432078 learned_addr learned my addr 172.21.15.138:0/1930432078 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.987+0000 7ff3df42b640 1 -- 172.21.15.138:0/1930432078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e2c76c0 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3df42b640 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55df7f17eea0 tx=0x55df7e3d4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3dc425640 1 -- 172.21.15.138:0/1930432078 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2dbe40 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3dc425640 1 -- 172.21.15.138:0/1930432078 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7f194000 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55df7e2c71e0 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3dc425640 1 -- 172.21.15.138:0/1930432078 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2db4a0 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df7e334ea0 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3dc425640 1 -- 172.21.15.138:0/1930432078 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55df7e2ed600 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3e0c86880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.988+0000 7ff3e0c86880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7e302c80 con 0x55df7f0bb800 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dc425640 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f218000 0x55df7e5a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dc425640 1 -- 172.21.15.138:0/1930432078 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55df7e2d5a40 con 0x55df7e2c3000 2024-04-27T05:21:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3df42b640 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f218000 0x55df7e5a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dfc2c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dfc2c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43876/0 (socket says 172.21.15.138:43876) 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3734872601 learned_addr learned my addr 172.21.15.138:0/3734872601 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3df42b640 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f218000 0x55df7e5a5080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55df7f1c8450 tx=0x55df7e3d54a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3734872601 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e334d00 con 0x55df7f0bb800 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/3734872601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55df7f17fb30 tx=0x55df7e3d5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28ad62527baa09e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3d9c20640 1 -- 172.21.15.138:0/3734872601 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2dbe40 con 0x55df7f0bb800 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3d9c20640 1 -- 172.21.15.138:0/3734872601 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7f194300 con 0x55df7f0bb800 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.989+0000 7ff3d9c20640 1 -- 172.21.15.138:0/3734872601 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2db4a0 con 0x55df7f0bb800 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3734872601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 msgr2=0x55df7e5a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3734872601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55df7f17fb30 tx=0x55df7e3d5c20 comp rx=0 tx=0).stop 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3734872601 shutdown_connections 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3734872601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3734872601 >> 172.21.15.138:0/3734872601 conn(0x55df7e2c3400 msgr2=0x55df7e3cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3734872601 shutdown_connections 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3734872601 wait complete. 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3e0c86880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7e2e6280 con 0x55df7f0bb800 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3dfc2c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3dfc2c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43880/0 (socket says 172.21.15.138:43880) 2024-04-27T05:21:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.990+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 learned_addr learned my addr 172.21.15.138:0/3751158998 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e334b60 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55df7f17f500 tx=0x55df7e3d5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3d841d640 1 -- 172.21.15.138:0/3751158998 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2dbe40 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3d841d640 1 -- 172.21.15.138:0/3751158998 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7f194480 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55df7e3349c0 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3d841d640 1 -- 172.21.15.138:0/3751158998 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7e2db4a0 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.991+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df7e334820 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3d841d640 1 -- 172.21.15.138:0/3751158998 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55df7e2ed8c0 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3d841d640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7e652000 0x55df7e5a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3d841d640 1 -- 172.21.15.138:0/3751158998 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55df7e2d5c00 con 0x55df7f0bb800 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7e652000 0x55df7e5a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55df7f0bbc00 0x55df7e5a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652400 con 0x55df7f0bbc00 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55df7f0bbc00 0x55df7e5a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7e652000 0x55df7e5a5600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55df7e3cec00 tx=0x55df7e61a120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.992+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55df7f0bbc00 0x55df7e5a5b80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 realm 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652800 con 0x55df7f0bbc00 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652c00 con 0x55df7f0bbc00 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653000 con 0x55df7f0bbc00 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55df7e653400 0x55df7e5a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652400 con 0x55df7e653400 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.993+0000 7ff3df42b640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55df7e653400 0x55df7e5a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3df42b640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55df7e653400 0x55df7e5a6100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3e0c86880 20 searching for the correct realm 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652800 con 0x55df7e653400 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3df42b640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55df7e652800 0x55df7e5a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652400 con 0x55df7e652800 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.994+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55df7e652800 0x55df7e5a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.995+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55df7e652800 0x55df7e5a6c00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.995+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.995+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55df7e652400 0x55df7e5a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.995+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.995+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55df7e652400 0x55df7e5a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.995+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55df7e652400 0x55df7e5a7180 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.996+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652800 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.997+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.997+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.997+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.997+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.997+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.997+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652800 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.998+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.999+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.999+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.999+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.999+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.999+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:36.999+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652800 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.000+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.001+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.001+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.001+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.001+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.001+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.001+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652800 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e653400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652400 2024-04-27T05:21:37.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.002+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7f0bbc00 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0ba800 con 0x55df7e652800 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652c00 con 0x55df7f0bbc00 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653000 con 0x55df7f0bbc00 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.003+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653800 con 0x55df7e653400 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 zone default found 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 4 Realm: () 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 zonegroup default 2024-04-27T05:21:37.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653c00 con 0x55df7e652400 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6000 con 0x55df7e653400 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6400 con 0x55df7e653400 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 started sync module instance, tier type = 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.004+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6800 con 0x55df7e652800 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.006+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.006+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94418385881344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6c00 con 0x55df7e652800 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.006+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a7000 con 0x55df7f0bbc00 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.006+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94418385881344] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.007+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.007+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94418385882496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a7400 con 0x55df7f0bbc00 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.007+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a7800 con 0x55df7e653400 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.007+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94418385882496] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.008+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.008+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94418385883648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6000 con 0x55df7e653400 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.008+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6400 con 0x55df7e652800 2024-04-27T05:21:37.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.009+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94418385883648] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.009+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.009+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94418385884800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a7800 con 0x55df7e652800 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.009+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653800 con 0x55df7f0bbc00 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.010+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.010+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94418385885952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e652c00 con 0x55df7f0bbc00 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.010+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653000 con 0x55df7e652400 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.010+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94418385884800] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.011+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.011+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94418385887104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e653c00 con 0x55df7e652400 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.011+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94418385885952] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.011+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f218c00 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.012+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94418385887104] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.012+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.012+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94418385888256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6800 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.012+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6c00 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94418385888256] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94418385889408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a6400 con 0x55df7e652400 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=0 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=1 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=2 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=3 2024-04-27T05:21:37.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=4 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=5 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.013+0000 7ff3e0c86880 20 add_watcher() i=6 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.014+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94418385889408] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.014+0000 7ff3e0c86880 20 add_watcher() i=7 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.014+0000 7ff3e0c86880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.014+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a7800 con 0x55df7f0bbc00 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.014+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.014+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94418385890560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7f0a7000 con 0x55df7f0bbc00 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94418385890560] v29'104 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3c63f9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3e0c86880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3e0c86880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3e0c86880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.015+0000 7ff3e0c86880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7e2e68c0 con 0x55df7f0a7c00 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3dec2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3dec2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43884/0 (socket says 172.21.15.138:43884) 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3dec2a640 1 -- 172.21.15.138:0/475289966 learned_addr learned my addr 172.21.15.138:0/475289966 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3dec2a640 1 -- 172.21.15.138:0/475289966 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e334680 con 0x55df7f0a7c00 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/475289966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55df7f240780 tx=0x55df7e61a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c08d2051e3d8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3c5bf8640 1 -- 172.21.15.138:0/475289966 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7f0a8580 con 0x55df7f0a7c00 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3c5bf8640 1 -- 172.21.15.138:0/475289966 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7f194000 con 0x55df7f0a7c00 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.016+0000 7ff3c5bf8640 1 -- 172.21.15.138:0/475289966 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7f0a86e0 con 0x55df7f0a7c00 2024-04-27T05:21:37.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- 172.21.15.138:0/475289966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 msgr2=0x55df7e5a6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/475289966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55df7f240780 tx=0x55df7e61a0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- 172.21.15.138:0/475289966 shutdown_connections 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/475289966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- 172.21.15.138:0/475289966 >> 172.21.15.138:0/475289966 conn(0x55df7f0a7400 msgr2=0x55df7e3cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- 172.21.15.138:0/475289966 shutdown_connections 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- 172.21.15.138:0/475289966 wait complete. 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 Processor -- start 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- start start 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3e0c86880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7e2e6780 con 0x55df7f0a7c00 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3dec2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3dec2a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43888/0 (socket says 172.21.15.138:43888) 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.017+0000 7ff3dec2a640 1 -- 172.21.15.138:0/2924581658 learned_addr learned my addr 172.21.15.138:0/2924581658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3dec2a640 1 -- 172.21.15.138:0/2924581658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df7e3344e0 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3dec2a640 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55df7e635740 tx=0x55df7e61a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3c43f5640 1 -- 172.21.15.138:0/2924581658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7f0a8840 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3c43f5640 1 -- 172.21.15.138:0/2924581658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55df7f194780 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55df7e2c6ea0 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3c43f5640 1 -- 172.21.15.138:0/2924581658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55df7f0a8580 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.018+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df7e2c76c0 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3c43f5640 1 -- 172.21.15.138:0/2924581658 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55df7e2ecb00 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3c43f5640 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f0a6800 0x55df7e5a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3c43f5640 1 -- 172.21.15.138:0/2924581658 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55df7e2d5dc0 con 0x55df7f0a7c00 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3e0c86880 5 note: GC not initialized 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f0a6800 0x55df7e5a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3c1bf0640 20 reqs_thread_entry: start 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3e0c86880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3dfc2c640 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f0a6800 0x55df7e5a7700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55df7f240bd0 tx=0x55df7e61a960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3e0c86880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3bfbec640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:07 2024 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3bf3ca640 20 reqs_thread_entry: start 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.019+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736000 con 0x55df7f0bbc00 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736400 con 0x55df7e652800 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.020+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736800 con 0x55df7e652800 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736c00 con 0x55df7e652800 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 4.34 4:2e3daf19:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737000 con 0x55df7e652800 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.35 4:adc4a58e:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737400 con 0x55df7f0bbc00 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 4.2c 4:346a3ea0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737800 con 0x55df7f0bbc00 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 4.2e 4:74cdb6e4:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737c00 con 0x55df7e653400 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 4.3a 4:5decd619:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74c000 con 0x55df7e653400 2024-04-27T05:21:37.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 4.c 4:335b21e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74c400 con 0x55df7f0bbc00 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.021+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74c800 con 0x55df7f0bbc00 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.2f 4:f6a773d6:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74cc00 con 0x55df7f0bbc00 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.2 4:4123e525:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74d000 con 0x55df7f0bbc00 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 4.3e 4:7d356189:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74d400 con 0x55df7e652800 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 4.16 4:6823909f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74d800 con 0x55df7e653400 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(76 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.022+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.023+0000 7ff3c63f9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.023+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94418385890560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74dc00 con 0x55df7f0bbc00 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.024+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94418385890560] v29'105 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.024+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94418385881344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e74d400 con 0x55df7e652800 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.025+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94418385881344] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.025+0000 7ff3e0c86880 20 remove_watcher() i=0 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.025+0000 7ff3e0c86880 2 removed watcher, disabling cache 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.025+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94418385882496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736400 con 0x55df7f0bbc00 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.026+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94418385882496] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.026+0000 7ff3e0c86880 20 remove_watcher() i=1 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.026+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94418385883648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736800 con 0x55df7e653400 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.027+0000 7ff3df42b640 1 -- 172.21.15.138:0/3751158998 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94418385883648] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668fc0 con 0x55df7e653400 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.027+0000 7ff3e0c86880 20 remove_watcher() i=2 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.027+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94418385884800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736c00 con 0x55df7e652800 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.028+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94418385884800] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e669200 con 0x55df7e652800 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.028+0000 7ff3e0c86880 20 remove_watcher() i=3 2024-04-27T05:21:37.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.028+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94418385885952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737000 con 0x55df7f0bbc00 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.029+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94418385885952] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7f0bbc00 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.029+0000 7ff3e0c86880 20 remove_watcher() i=4 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.029+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94418385887104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e736000 con 0x55df7e652400 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.030+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94418385887104] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.030+0000 7ff3e0c86880 20 remove_watcher() i=5 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.030+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94418385888256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737400 con 0x55df7e652400 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.031+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94418385888256] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.031+0000 7ff3e0c86880 20 remove_watcher() i=6 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.031+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94418385889408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55df7e737800 con 0x55df7e652400 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3dfc2c640 1 -- 172.21.15.138:0/3751158998 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94418385889408] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55df7e668900 con 0x55df7e652400 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 20 remove_watcher() i=7 2024-04-27T05:21:37.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55df7e652400 msgr2=0x55df7e5a7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55df7e652400 0x55df7e5a7180 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55df7f0bbc00 msgr2=0x55df7e5a5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55df7f0bbc00 0x55df7e5a5b80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55df7e652800 msgr2=0x55df7e5a6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55df7e652800 0x55df7e5a6c00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55df7e653400 msgr2=0x55df7e5a6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55df7e653400 0x55df7e5a6100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7e652000 msgr2=0x55df7e5a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7e652000 0x55df7e5a5600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55df7e3cec00 tx=0x55df7e61a120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 msgr2=0x55df7e5a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/3751158998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0bb800 0x55df7e5a4b00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55df7f17f500 tx=0x55df7e3d5a40 comp rx=0 tx=0).stop 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 reap_dead start 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3dec2a640 1 -- 172.21.15.138:0/3751158998 reap_dead start 2024-04-27T05:21:37.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 shutdown_connections 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 >> 172.21.15.138:0/3751158998 conn(0x55df7e2c3400 msgr2=0x55df7e3cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 shutdown_connections 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/3751158998 wait complete. 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f0a6800 msgr2=0x55df7e5a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f0a6800 0x55df7e5a7700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55df7f240bd0 tx=0x55df7e61a960 comp rx=0 tx=0).stop 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 msgr2=0x55df7e5a6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55df7e635740 tx=0x55df7e61a000 comp rx=0 tx=0).stop 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 shutdown_connections 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f0a6800 0x55df7e5a7700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/2924581658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7f0a7c00 0x55df7e5a6680 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 >> 172.21.15.138:0/2924581658 conn(0x55df7f0a7400 msgr2=0x55df7e3cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 shutdown_connections 2024-04-27T05:21:37.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.032+0000 7ff3e0c86880 1 -- 172.21.15.138:0/2924581658 wait complete. 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f218000 msgr2=0x55df7e5a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f218000 0x55df7e5a5080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55df7f1c8450 tx=0x55df7e3d54a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 msgr2=0x55df7e5a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55df7f17eea0 tx=0x55df7e3d4c60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 shutdown_connections 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55df7f218000 0x55df7e5a5080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 --2- 172.21.15.138:0/1930432078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55df7e2c3000 0x55df7e5a4580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 >> 172.21.15.138:0/1930432078 conn(0x55df7e2c2c00 msgr2=0x55df7e3cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 shutdown_connections 2024-04-27T05:21:37.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.033+0000 7ff3e0c86880 1 -- 172.21.15.138:0/1930432078 wait complete. 2024-04-27T05:21:37.062 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-04-27T05:21:37.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:37.062 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:21:37.062 INFO:teuthology.orchestra.run.smithi138.stdout: "num_shards": 11, 2024-04-27T05:21:37.062 INFO:teuthology.orchestra.run.smithi138.stdout: "tenant": "", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "zonegroup": "7c1a7efe-08a0-41e8-a0e4-e18cf496a33b", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_rule": "default-placement", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "explicit_placement": { 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "" 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "marker": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": "Normal", 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "versioned": false, 2024-04-27T05:21:37.063 INFO:teuthology.orchestra.run.smithi138.stdout: "versioning_enabled": false, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "object_lock_enabled": false, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_enabled": false, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "mtime": "2024-04-27T05:21:36.332917Z", 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "creation_time": "2024-04-27T05:21:36.328858Z", 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "usage": {}, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:37.064 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:37.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.065 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-04-27T05:21:37.065 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:21:36.332917Z', 'creation_time': '2024-04-27T05:21:36.328858Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-27T05:21:37.072 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T05:21:37.072 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T05:21:37.072 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T05:21:37.072 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T05:21:37.145 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:37.145 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:37.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b0733c880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b0733c880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55559d9ae600 con 0x55559d6d7400 2024-04-27T05:21:37.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b0733c880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55559d716280 con 0x55559d6d7000 2024-04-27T05:21:37.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b03d3f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b0353e640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55559d6d7400 0x55559d7de800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60940/0 (socket says 172.21.15.138:60940) 2024-04-27T05:21:37.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b03d3f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43902/0 (socket says 172.21.15.138:43902) 2024-04-27T05:21:37.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b03d3f640 1 -- 172.21.15.138:0/1062007066 learned_addr learned my addr 172.21.15.138:0/1062007066 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.143+0000 7f8b03d3f640 1 -- 172.21.15.138:0/1062007066 >> v1:172.21.15.114:6789/0 conn(0x55559d6d7400 legacy=0x55559d7de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.144+0000 7f8b03d3f640 1 -- 172.21.15.138:0/1062007066 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d6db380 con 0x55559d6d7000 2024-04-27T05:21:37.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.144+0000 7f8b02d3d640 1 -- 172.21.15.138:0/1062007066 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829841565 0 0) 0x55559d6da9c0 con 0x55559d6d7400 2024-04-27T05:21:37.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.145+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/1062007066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55559d7e2d50 tx=0x55559d7e8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b511f88e8979cde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.145+0000 7f8b02d3d640 1 -- 172.21.15.138:0/1062007066 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6efce0 con 0x55559d6d7000 2024-04-27T05:21:37.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.145+0000 7f8b02d3d640 1 -- 172.21.15.138:0/1062007066 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559d9afb00 con 0x55559d6d7000 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.145+0000 7f8b02d3d640 1 -- 172.21.15.138:0/1062007066 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6efe40 con 0x55559d6d7000 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 -- 172.21.15.138:0/1062007066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 msgr2=0x55559d9b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 --2- 172.21.15.138:0/1062007066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55559d7e2d50 tx=0x55559d7e8de0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 -- 172.21.15.138:0/1062007066 shutdown_connections 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 --2- 172.21.15.138:0/1062007066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 -- 172.21.15.138:0/1062007066 >> 172.21.15.138:0/1062007066 conn(0x55559d6d6c00 msgr2=0x55559d7de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 -- 172.21.15.138:0/1062007066 shutdown_connections 2024-04-27T05:21:37.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 -- 172.21.15.138:0/1062007066 wait complete. 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.146+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b0733c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b0733c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d716780 con 0x55559d6d7000 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b03d3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b03d3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43914/0 (socket says 172.21.15.138:43914) 2024-04-27T05:21:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b03d3f640 1 -- 172.21.15.138:0/978098772 learned_addr learned my addr 172.21.15.138:0/978098772 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b03d3f640 1 -- 172.21.15.138:0/978098772 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d6daea0 con 0x55559d6d7000 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/978098772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55559e4dd590 tx=0x55559d7e8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317da55f99f2622b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b0253c640 1 -- 172.21.15.138:0/978098772 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6ef4a0 con 0x55559d6d7000 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b0253c640 1 -- 172.21.15.138:0/978098772 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559d9afe00 con 0x55559d6d7000 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.147+0000 7f8b0253c640 1 -- 172.21.15.138:0/978098772 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6efce0 con 0x55559d6d7000 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- 172.21.15.138:0/978098772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 msgr2=0x55559d9b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 --2- 172.21.15.138:0/978098772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55559e4dd590 tx=0x55559d7e8cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- 172.21.15.138:0/978098772 shutdown_connections 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 --2- 172.21.15.138:0/978098772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- 172.21.15.138:0/978098772 >> 172.21.15.138:0/978098772 conn(0x55559d6d6c00 msgr2=0x55559d7df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- 172.21.15.138:0/978098772 shutdown_connections 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- 172.21.15.138:0/978098772 wait complete. 2024-04-27T05:21:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b0733c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d7168c0 con 0x55559d6d7000 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b03d3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b03d3f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43922/0 (socket says 172.21.15.138:43922) 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b03d3f640 1 -- 172.21.15.138:0/2892666490 learned_addr learned my addr 172.21.15.138:0/2892666490 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.148+0000 7f8b03d3f640 1 -- 172.21.15.138:0/2892666490 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d6db6c0 con 0x55559d6d7000 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55559e5932c0 tx=0x55559d7e8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b00d39640 1 -- 172.21.15.138:0/2892666490 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6efe40 con 0x55559d6d7000 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b00d39640 1 -- 172.21.15.138:0/2892666490 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559e4b6000 con 0x55559d6d7000 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b00d39640 1 -- 172.21.15.138:0/2892666490 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6ef4a0 con 0x55559d6d7000 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55559d6db1e0 con 0x55559d6d7000 2024-04-27T05:21:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55559d748ea0 con 0x55559d6d7000 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.149+0000 7f8b0733c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b0733c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d716c80 con 0x55559e4cec00 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b00d39640 1 -- 172.21.15.138:0/2892666490 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55559d700b00 con 0x55559d6d7000 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b00d39640 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e4cfc00 0x55559d9b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b00d39640 1 -- 172.21.15.138:0/2892666490 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55559d6e9a40 con 0x55559d6d7000 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b04540640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b04540640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43936/0 (socket says 172.21.15.138:43936) 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b04540640 1 -- 172.21.15.138:0/2436069564 learned_addr learned my addr 172.21.15.138:0/2436069564 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e4cfc00 0x55559d9b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b04540640 1 -- 172.21.15.138:0/2436069564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d748d00 con 0x55559e4cec00 2024-04-27T05:21:37.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e4cfc00 0x55559d9b9080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55559e5c8780 tx=0x55559d7e8cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8b04540640 1 --2- 172.21.15.138:0/2436069564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55559e593e60 tx=0x55559d7e9c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9933426383cc83f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8afe534640 1 -- 172.21.15.138:0/2436069564 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6efe40 con 0x55559e4cec00 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8afe534640 1 -- 172.21.15.138:0/2436069564 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559e4b6600 con 0x55559e4cec00 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.150+0000 7f8afe534640 1 -- 172.21.15.138:0/2436069564 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6ef4a0 con 0x55559e4cec00 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- 172.21.15.138:0/2436069564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 msgr2=0x55559d9b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 --2- 172.21.15.138:0/2436069564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55559e593e60 tx=0x55559d7e9c20 comp rx=0 tx=0).stop 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- 172.21.15.138:0/2436069564 shutdown_connections 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 --2- 172.21.15.138:0/2436069564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- 172.21.15.138:0/2436069564 >> 172.21.15.138:0/2436069564 conn(0x55559d6d7400 msgr2=0x55559d7e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- 172.21.15.138:0/2436069564 shutdown_connections 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- 172.21.15.138:0/2436069564 wait complete. 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b0733c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d6fa280 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b04540640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b04540640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43952/0 (socket says 172.21.15.138:43952) 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.151+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 learned_addr learned my addr 172.21.15.138:0/466561460 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d748b60 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8b04540640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55559e592b40 tx=0x55559d7e91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8afcd31640 1 -- 172.21.15.138:0/466561460 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6efe40 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8afcd31640 1 -- 172.21.15.138:0/466561460 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559e4b6780 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55559d7489c0 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8afcd31640 1 -- 172.21.15.138:0/466561460 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559d6ef4a0 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.152+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55559d748820 con 0x55559e4cec00 2024-04-27T05:21:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8afcd31640 1 -- 172.21.15.138:0/466561460 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55559d701600 con 0x55559e4cec00 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8afcd31640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559da54000 0x55559d9b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8afcd31640 1 -- 172.21.15.138:0/466561460 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55559d6e9c00 con 0x55559e4cec00 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55559e4cf000 0x55559d9b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b0353e640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559da54000 0x55559d9b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf400 con 0x55559e4cf000 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b04540640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55559e4cf000 0x55559d9b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b0353e640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559da54000 0x55559d9b9600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55559d7e33e0 tx=0x55559da1c120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b04540640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55559e4cf000 0x55559d9b9b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.153+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 realm 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf800 con 0x55559e4cf000 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55000 con 0x55559e4cf000 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55400 con 0x55559e4cf000 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55559da55800 0x55559d9ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55c00 con 0x55559da55800 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.154+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55559da55800 0x55559d9ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55559da55800 0x55559d9ba100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b0733c880 20 searching for the correct realm 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55000 con 0x55559da55800 2024-04-27T05:21:37.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b03d3f640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55559da55000 0x55559d9bac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55c00 con 0x55559da55000 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.155+0000 7f8b0353e640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55559da55000 0x55559d9bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.156+0000 7f8b0353e640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55559da55000 0x55559d9bac00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.156+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.156+0000 7f8b0353e640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55559da55c00 0x55559d9bb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.156+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.156+0000 7f8b04540640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55559da55c00 0x55559d9bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.156+0000 7f8b04540640 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55559da55c00 0x55559d9bb180 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.157+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.158+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.158+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.158+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.158+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.158+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.158+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55000 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.159+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.160+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55000 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.161+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55000 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.162+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55800 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55c00 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559e4cf000 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da54c00 con 0x55559da55000 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0733c880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0733c880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0733c880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0733c880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.163+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55400 con 0x55559e4cf000 2024-04-27T05:21:37.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf400 con 0x55559e4cf000 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf800 con 0x55559da55800 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 zone default found 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 4 Realm: () 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 zonegroup default 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e000 con 0x55559da55c00 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.164+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e400 con 0x55559da55800 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e800 con 0x55559da55800 2024-04-27T05:21:37.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 started sync module instance, tier type = 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.165+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7ec00 con 0x55559da55000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.166+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.166+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93826204764416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7f000 con 0x55559da55000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.166+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7f400 con 0x55559e4cf000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.167+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93826204764416] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.167+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.167+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93826204765568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7f800 con 0x55559e4cf000 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.167+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7fc00 con 0x55559da55800 2024-04-27T05:21:37.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.168+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93826204765568] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.168+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.168+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93826204766720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e400 con 0x55559da55800 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.168+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e800 con 0x55559da55000 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.169+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.169+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93826204767872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7fc00 con 0x55559da55000 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.169+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93826204766720] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.169+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e000 con 0x55559e4cf000 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.170+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93826204767872] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.170+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.170+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93826204769024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf800 con 0x55559e4cf000 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.170+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf400 con 0x55559da55c00 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.171+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93826204769024] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.171+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.171+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93826204770176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da55400 con 0x55559da55c00 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.171+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e635800 con 0x55559da55c00 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.172+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93826204770176] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.172+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.172+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93826204771328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e635400 con 0x55559da55c00 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.172+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7ec00 con 0x55559da55c00 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93826204772480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7f000 con 0x55559da55c00 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=0 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=1 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=2 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=3 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=4 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=5 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93826204771328] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.173+0000 7f8b0733c880 20 add_watcher() i=6 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.174+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93826204772480] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.174+0000 7f8b0733c880 20 add_watcher() i=7 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.174+0000 7f8b0733c880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.174+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e800 con 0x55559e4cf000 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.174+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.174+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93826204773632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7f400 con 0x55559e4cf000 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.175+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93826204773632] v29'106 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.175+0000 7f8aead0d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.175+0000 7f8b0733c880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.175+0000 7f8b0733c880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0733c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0733c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d6fa8c0 con 0x55559da54c00 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0353e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0353e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43964/0 (socket says 172.21.15.138:43964) 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0353e640 1 -- 172.21.15.138:0/713572659 learned_addr learned my addr 172.21.15.138:0/713572659 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0353e640 1 -- 172.21.15.138:0/713572659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d748680 con 0x55559da54c00 2024-04-27T05:21:37.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.176+0000 7f8b0353e640 1 --2- 172.21.15.138:0/713572659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55559da39620 tx=0x55559da1c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1678f8dc667ec36 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8aea50c640 1 -- 172.21.15.138:0/713572659 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559da80580 con 0x55559da54c00 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8aea50c640 1 -- 172.21.15.138:0/713572659 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559e4b6000 con 0x55559da54c00 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8aea50c640 1 -- 172.21.15.138:0/713572659 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559da806e0 con 0x55559da54c00 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 -- 172.21.15.138:0/713572659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 msgr2=0x55559d9ba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 --2- 172.21.15.138:0/713572659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x55559da39620 tx=0x55559da1c180 comp rx=0 tx=0).stop 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 -- 172.21.15.138:0/713572659 shutdown_connections 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 --2- 172.21.15.138:0/713572659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 -- 172.21.15.138:0/713572659 >> 172.21.15.138:0/713572659 conn(0x55559da7f800 msgr2=0x55559d7e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 -- 172.21.15.138:0/713572659 shutdown_connections 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.177+0000 7f8b0733c880 1 -- 172.21.15.138:0/713572659 wait complete. 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0733c880 1 Processor -- start 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0733c880 1 -- start start 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0733c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0733c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d6fa780 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0353e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0353e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43976/0 (socket says 172.21.15.138:43976) 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0353e640 1 -- 172.21.15.138:0/1652584341 learned_addr learned my addr 172.21.15.138:0/1652584341 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0353e640 1 -- 172.21.15.138:0/1652584341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d7484e0 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0353e640 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55559da96630 tx=0x55559da1cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8ae8d09640 1 -- 172.21.15.138:0/1652584341 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559da80840 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8ae8d09640 1 -- 172.21.15.138:0/1652584341 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55559e4b6a80 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55559d6db380 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.178+0000 7f8ae8d09640 1 -- 172.21.15.138:0/1652584341 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55559da80580 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.179+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55559d6daea0 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.179+0000 7f8ae8d09640 1 -- 172.21.15.138:0/1652584341 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55559d701b80 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.179+0000 7f8ae8d09640 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e635800 0x55559d9bb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.179+0000 7f8ae8d09640 1 -- 172.21.15.138:0/1652584341 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55559d6e9dc0 con 0x55559da54c00 2024-04-27T05:21:37.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.179+0000 7f8b0733c880 5 note: GC not initialized 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.179+0000 7f8b04540640 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e635800 0x55559d9bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8ae6504640 20 reqs_thread_entry: start 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b04540640 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e635800 0x55559d9bb700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55559e642b70 tx=0x55559da1c8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8ae4500640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:07 2024 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8ae3cde640 20 reqs_thread_entry: start 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e000 con 0x55559da55000 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.180+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e400 con 0x55559da55000 2024-04-27T05:21:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e800 con 0x55559da55000 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4ec00 con 0x55559da55000 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(65 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:37.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 4.34 4:2e3daf19:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4f000 con 0x55559da55000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.35 4:adc4a58e:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4f400 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 4.2c 4:346a3ea0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4f800 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 4.2e 4:74cdb6e4:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4fc00 con 0x55559da55800 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.181+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 4.3a 4:5decd619:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e000 con 0x55559da55800 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 4.c 4:335b21e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e400 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e800 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.2f 4:f6a773d6:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4ec00 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.2 4:4123e525:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e4cf400 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 4.3e 4:7d356189:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7e000 con 0x55559da55000 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 4.16 4:6823909f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4f400 con 0x55559da55800 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(76 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.182+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.183+0000 7f8aead0d640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.183+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93826204773632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4f800 con 0x55559e4cf000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.184+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 93826204773632] v29'107 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.184+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93826204764416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e400 con 0x55559da55000 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.185+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 93826204764416] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.185+0000 7f8b0733c880 20 remove_watcher() i=0 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.185+0000 7f8b0733c880 2 removed watcher, disabling cache 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.185+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93826204765568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4e800 con 0x55559e4cf000 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.186+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 93826204765568] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.186+0000 7f8b0733c880 20 remove_watcher() i=1 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.186+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93826204766720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559db4ec00 con 0x55559da55800 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.187+0000 7f8b03d3f640 1 -- 172.21.15.138:0/466561460 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 93826204766720] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6cfc0 con 0x55559da55800 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.187+0000 7f8b0733c880 20 remove_watcher() i=2 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.187+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93826204767872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7ec00 con 0x55559da55000 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.187+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 93826204767872] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6d200 con 0x55559da55000 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.187+0000 7f8b0733c880 20 remove_watcher() i=3 2024-04-27T05:21:37.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.188+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93826204769024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559da7f000 con 0x55559e4cf000 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.189+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 93826204769024] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559e4cf000 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.189+0000 7f8b0733c880 20 remove_watcher() i=4 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.189+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93826204770176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e638000 con 0x55559da55c00 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.189+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 93826204770176] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.189+0000 7f8b0733c880 20 remove_watcher() i=5 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.189+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93826204771328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e638400 con 0x55559da55c00 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.190+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 93826204771328] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.190+0000 7f8b0733c880 20 remove_watcher() i=6 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.190+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93826204772480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55559e638800 con 0x55559da55c00 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b04540640 1 -- 172.21.15.138:0/466561460 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 93826204772480] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55559da6c900 con 0x55559da55c00 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 20 remove_watcher() i=7 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55559da55c00 msgr2=0x55559d9bb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55559da55c00 0x55559d9bb180 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55559e4cf000 msgr2=0x55559d9b9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55559e4cf000 0x55559d9b9b80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55559da55000 msgr2=0x55559d9bac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55559da55000 0x55559d9bac00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55559da55800 msgr2=0x55559d9ba100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55559da55800 0x55559d9ba100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559da54000 msgr2=0x55559d9b9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559da54000 0x55559d9b9600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55559d7e33e0 tx=0x55559da1c120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 msgr2=0x55559d9b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 --2- 172.21.15.138:0/466561460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559e4cec00 0x55559d9b8b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55559e592b40 tx=0x55559d7e91a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 reap_dead start 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0353e640 1 -- 172.21.15.138:0/466561460 reap_dead start 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 shutdown_connections 2024-04-27T05:21:37.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 >> 172.21.15.138:0/466561460 conn(0x55559d6d7400 msgr2=0x55559d7e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 shutdown_connections 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.191+0000 7f8b0733c880 1 -- 172.21.15.138:0/466561460 wait complete. 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e635800 msgr2=0x55559d9bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e635800 0x55559d9bb700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55559e642b70 tx=0x55559da1c8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 msgr2=0x55559d9ba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55559da96630 tx=0x55559da1cae0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 shutdown_connections 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e635800 0x55559d9bb700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/1652584341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559da54c00 0x55559d9ba680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 >> 172.21.15.138:0/1652584341 conn(0x55559da7f800 msgr2=0x55559d7e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 shutdown_connections 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/1652584341 wait complete. 2024-04-27T05:21:37.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e4cfc00 msgr2=0x55559d9b9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e4cfc00 0x55559d9b9080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55559e5c8780 tx=0x55559d7e8cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 msgr2=0x55559d9b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55559e5932c0 tx=0x55559d7e8c60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 shutdown_connections 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55559e4cfc00 0x55559d9b9080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 --2- 172.21.15.138:0/2892666490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55559d6d7000 0x55559d9b8580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 >> 172.21.15.138:0/2892666490 conn(0x55559d6d6c00 msgr2=0x55559d7df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 shutdown_connections 2024-04-27T05:21:37.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.192+0000 7f8b0733c880 1 -- 172.21.15.138:0/2892666490 wait complete. 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "num_shards": 11, 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tenant": "", 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "zonegroup": "7c1a7efe-08a0-41e8-a0e4-e18cf496a33b", 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_rule": "default-placement", 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "explicit_placement": { 2024-04-27T05:21:37.220 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "" 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "marker": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": "Normal", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "versioned": false, 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "versioning_enabled": false, 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "object_lock_enabled": false, 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_enabled": false, 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "mtime": "2024-04-27T05:21:36.332917Z", 2024-04-27T05:21:37.221 INFO:teuthology.orchestra.run.smithi138.stdout: "creation_time": "2024-04-27T05:21:36.328858Z", 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "usage": { 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw.main": { 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 3, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "size_actual": 4096, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "size_utilized": 3, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb": 1, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb_actual": 4, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb_utilized": 1, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:37.222 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:37.223 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:37.223 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:37.223 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:37.223 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:37.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:37.223 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:21:36.332917Z', 'creation_time': '2024-04-27T05:21:36.328858Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T05:21:37.223 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-27T05:21:37.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-27T05:21:37.224 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-04-27T05:21:37.304 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:37.304 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:37.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.302+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.302+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.302+0000 7f6939212880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.302+0000 7f6939212880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55edd4584600 con 0x55edd42ad400 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.302+0000 7f6939212880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55edd42ec280 con 0x55edd42ad000 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.302+0000 7f69356b2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.303+0000 7f69356b2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:43986/0 (socket says 172.21.15.138:43986) 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.303+0000 7f69356b2640 1 -- 172.21.15.138:0/2478409404 learned_addr learned my addr 172.21.15.138:0/2478409404 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.303+0000 7f6934eb1640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55edd42ad400 0x55edd43b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60952/0 (socket says 172.21.15.138:60952) 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.303+0000 7f69356b2640 1 -- 172.21.15.138:0/2478409404 >> v1:172.21.15.114:6789/0 conn(0x55edd42ad400 legacy=0x55edd43b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.303+0000 7f69356b2640 1 -- 172.21.15.138:0/2478409404 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd42b1380 con 0x55edd42ad000 2024-04-27T05:21:37.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.304+0000 7f69356b2640 1 --2- 172.21.15.138:0/2478409404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55edd43b8d50 tx=0x55edd43bede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c759f556aab8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.304+0000 7f69346b0640 1 -- 172.21.15.138:0/2478409404 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c5ce0 con 0x55edd42ad000 2024-04-27T05:21:37.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f69346b0640 1 -- 172.21.15.138:0/2478409404 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd4585b00 con 0x55edd42ad000 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f69346b0640 1 -- 172.21.15.138:0/2478409404 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c5e40 con 0x55edd42ad000 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 -- 172.21.15.138:0/2478409404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 msgr2=0x55edd458e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 --2- 172.21.15.138:0/2478409404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55edd43b8d50 tx=0x55edd43bede0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 -- 172.21.15.138:0/2478409404 shutdown_connections 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 --2- 172.21.15.138:0/2478409404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 -- 172.21.15.138:0/2478409404 >> 172.21.15.138:0/2478409404 conn(0x55edd42acc00 msgr2=0x55edd43b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 -- 172.21.15.138:0/2478409404 shutdown_connections 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.305+0000 7f6939212880 1 -- 172.21.15.138:0/2478409404 wait complete. 2024-04-27T05:21:37.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6939212880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6939212880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd42ec780 con 0x55edd42ad000 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f69356b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f69356b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44002/0 (socket says 172.21.15.138:44002) 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f69356b2640 1 -- 172.21.15.138:0/4142413774 learned_addr learned my addr 172.21.15.138:0/4142413774 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f69356b2640 1 -- 172.21.15.138:0/4142413774 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd42b0ea0 con 0x55edd42ad000 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f69356b2640 1 --2- 172.21.15.138:0/4142413774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55edd50b1560 tx=0x55edd43becc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c7fae4a934d304 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6933eaf640 1 -- 172.21.15.138:0/4142413774 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c54a0 con 0x55edd42ad000 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6933eaf640 1 -- 172.21.15.138:0/4142413774 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd4585e00 con 0x55edd42ad000 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.306+0000 7f6933eaf640 1 -- 172.21.15.138:0/4142413774 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c5ce0 con 0x55edd42ad000 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- 172.21.15.138:0/4142413774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 msgr2=0x55edd458e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 --2- 172.21.15.138:0/4142413774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55edd50b1560 tx=0x55edd43becc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- 172.21.15.138:0/4142413774 shutdown_connections 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 --2- 172.21.15.138:0/4142413774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- 172.21.15.138:0/4142413774 >> 172.21.15.138:0/4142413774 conn(0x55edd42acc00 msgr2=0x55edd43b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- 172.21.15.138:0/4142413774 shutdown_connections 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- 172.21.15.138:0/4142413774 wait complete. 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.307+0000 7f6939212880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd42ec8c0 con 0x55edd42ad000 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69356b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69356b2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44004/0 (socket says 172.21.15.138:44004) 2024-04-27T05:21:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69356b2640 1 -- 172.21.15.138:0/170029821 learned_addr learned my addr 172.21.15.138:0/170029821 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69356b2640 1 -- 172.21.15.138:0/170029821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd42b16c0 con 0x55edd42ad000 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69356b2640 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55edd51692c0 tx=0x55edd43bec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69326ac640 1 -- 172.21.15.138:0/170029821 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c5e40 con 0x55edd42ad000 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69326ac640 1 -- 172.21.15.138:0/170029821 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd508c000 con 0x55edd42ad000 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f69326ac640 1 -- 172.21.15.138:0/170029821 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c54a0 con 0x55edd42ad000 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55edd42b11e0 con 0x55edd42ad000 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edd431eea0 con 0x55edd42ad000 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.308+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6939212880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6939212880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd42ecc80 con 0x55edd50a4c00 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f69326ac640 1 -- 172.21.15.138:0/170029821 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55edd42d6b00 con 0x55edd42ad000 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6935eb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f69326ac640 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd50a5c00 0x55edd458f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6935eb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44010/0 (socket says 172.21.15.138:44010) 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6935eb3640 1 -- 172.21.15.138:0/2331912876 learned_addr learned my addr 172.21.15.138:0/2331912876 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f69326ac640 1 -- 172.21.15.138:0/170029821 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55edd42bfa40 con 0x55edd42ad000 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f69356b2640 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd50a5c00 0x55edd458f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6935eb3640 1 -- 172.21.15.138:0/2331912876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd431ed00 con 0x55edd50a4c00 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f69356b2640 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd50a5c00 0x55edd458f080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55edd5198600 tx=0x55edd43beea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.309+0000 7f6935eb3640 1 --2- 172.21.15.138:0/2331912876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55edd50b15f0 tx=0x55edd43bfc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cade852485c0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f692fea7640 1 -- 172.21.15.138:0/2331912876 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c5e40 con 0x55edd50a4c00 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f692fea7640 1 -- 172.21.15.138:0/2331912876 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd508c600 con 0x55edd50a4c00 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f692fea7640 1 -- 172.21.15.138:0/2331912876 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c54a0 con 0x55edd50a4c00 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- 172.21.15.138:0/2331912876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 msgr2=0x55edd458eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 --2- 172.21.15.138:0/2331912876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55edd50b15f0 tx=0x55edd43bfc20 comp rx=0 tx=0).stop 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- 172.21.15.138:0/2331912876 shutdown_connections 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 --2- 172.21.15.138:0/2331912876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- 172.21.15.138:0/2331912876 >> 172.21.15.138:0/2331912876 conn(0x55edd42ad400 msgr2=0x55edd43b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- 172.21.15.138:0/2331912876 shutdown_connections 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- 172.21.15.138:0/2331912876 wait complete. 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.310+0000 7f6939212880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd42d0280 con 0x55edd50a4c00 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6935eb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6935eb3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44024/0 (socket says 172.21.15.138:44024) 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 learned_addr learned my addr 172.21.15.138:0/3150561739 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd431eb60 con 0x55edd50a4c00 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6935eb3640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55edd51693e0 tx=0x55edd43bf9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f692e6a4640 1 -- 172.21.15.138:0/3150561739 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c5e40 con 0x55edd50a4c00 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f692e6a4640 1 -- 172.21.15.138:0/3150561739 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd508c780 con 0x55edd50a4c00 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55edd431e9c0 con 0x55edd50a4c00 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f692e6a4640 1 -- 172.21.15.138:0/3150561739 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd42c54a0 con 0x55edd50a4c00 2024-04-27T05:21:37.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.311+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edd431e820 con 0x55edd50a4c00 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f692e6a4640 1 -- 172.21.15.138:0/3150561739 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55edd42d7600 con 0x55edd50a4c00 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f692e6a4640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd462c000 0x55edd458f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f692e6a4640 1 -- 172.21.15.138:0/3150561739 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55edd42bfc00 con 0x55edd50a4c00 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f6934eb1640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd462c000 0x55edd458f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55edd50a5000 0x55edd458fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd50a5400 con 0x55edd50a5000 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.312+0000 7f6935eb3640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55edd50a5000 0x55edd458fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6934eb1640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd462c000 0x55edd458f600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55edd43b9380 tx=0x55edd45f0120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6935eb3640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55edd50a5000 0x55edd458fb80 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 realm 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd50a5800 con 0x55edd50a5000 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462d000 con 0x55edd50a5000 2024-04-27T05:21:37.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462d400 con 0x55edd50a5000 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.313+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55edd462d800 0x55edd4590100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462dc00 con 0x55edd462d800 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f69356b2640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55edd462d800 0x55edd4590100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f69356b2640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55edd462d800 0x55edd4590100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f6939212880 20 searching for the correct realm 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.314+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462d000 con 0x55edd462d800 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f69356b2640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55edd462d000 0x55edd4590c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462dc00 con 0x55edd462d000 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f6934eb1640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55edd462d000 0x55edd4590c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f6934eb1640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55edd462d000 0x55edd4590c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f6934eb1640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55edd462dc00 0x55edd4591180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.315+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.316+0000 7f6935eb3640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55edd462dc00 0x55edd4591180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.316+0000 7f6935eb3640 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55edd462dc00 0x55edd4591180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.316+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.316+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.316+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.316+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d000 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.317+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d000 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.318+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.319+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d000 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.320+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.321+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d000 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d800 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462dc00 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd50a5000 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.322+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462cc00 con 0x55edd462d000 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462d400 con 0x55edd50a5000 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd50a5400 con 0x55edd50a5000 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd50a5800 con 0x55edd462d800 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 zone default found 2024-04-27T05:21:37.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 4 Realm: () 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 zonegroup default 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.323+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650000 con 0x55edd462dc00 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650400 con 0x55edd462d800 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650800 con 0x55edd462d800 2024-04-27T05:21:37.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 started sync module instance, tier type = 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.324+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650c00 con 0x55edd462d000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.325+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.325+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94479958173952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651000 con 0x55edd462d000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.325+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651400 con 0x55edd50a5000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.326+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94479958173952] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.326+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.327+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94479958175104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651800 con 0x55edd50a5000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.327+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651c00 con 0x55edd462d800 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.328+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94479958175104] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.328+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.328+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94479958176256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650400 con 0x55edd462d800 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.328+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650800 con 0x55edd462d000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.329+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.329+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94479958177408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651c00 con 0x55edd462d000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.329+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650000 con 0x55edd50a5000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.329+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94479958176256] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.330+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94479958177408] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.330+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.330+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94479958178560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd50a5800 con 0x55edd50a5000 2024-04-27T05:21:37.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.330+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd50a5400 con 0x55edd462dc00 2024-04-27T05:21:37.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.331+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94479958178560] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.331+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.331+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94479958179712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd462d400 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.331+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd520ac00 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.332+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94479958179712] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.332+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.332+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94479958180864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd520a800 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.332+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650c00 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94479958182016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651000 con 0x55edd462dc00 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=0 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=1 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=2 2024-04-27T05:21:37.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=3 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=4 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=5 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94479958180864] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.333+0000 7f6939212880 20 add_watcher() i=6 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.334+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94479958182016] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.334+0000 7f6939212880 20 add_watcher() i=7 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.334+0000 7f6939212880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.334+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650800 con 0x55edd50a5000 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.334+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.334+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94479958183168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4651400 con 0x55edd50a5000 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.335+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94479958183168] v29'108 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.335+0000 7f691c680640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.335+0000 7f6939212880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.335+0000 7f6939212880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.335+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.335+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6939212880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6939212880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd42d08c0 con 0x55edd462cc00 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6934eb1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6934eb1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44040/0 (socket says 172.21.15.138:44040) 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6934eb1640 1 -- 172.21.15.138:0/1295366275 learned_addr learned my addr 172.21.15.138:0/1295366275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6934eb1640 1 -- 172.21.15.138:0/1295366275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd431e680 con 0x55edd462cc00 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f6934eb1640 1 --2- 172.21.15.138:0/1295366275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55edd46164b0 tx=0x55edd45f0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f877dc5fa86ea4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f691be7f640 1 -- 172.21.15.138:0/1295366275 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd4652580 con 0x55edd462cc00 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f691be7f640 1 -- 172.21.15.138:0/1295366275 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd508c000 con 0x55edd462cc00 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.336+0000 7f691be7f640 1 -- 172.21.15.138:0/1295366275 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd46526e0 con 0x55edd462cc00 2024-04-27T05:21:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 -- 172.21.15.138:0/1295366275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 msgr2=0x55edd4590680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 --2- 172.21.15.138:0/1295366275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55edd46164b0 tx=0x55edd45f0180 comp rx=0 tx=0).stop 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 -- 172.21.15.138:0/1295366275 shutdown_connections 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 --2- 172.21.15.138:0/1295366275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 -- 172.21.15.138:0/1295366275 >> 172.21.15.138:0/1295366275 conn(0x55edd4651800 msgr2=0x55edd43b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 -- 172.21.15.138:0/1295366275 shutdown_connections 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 -- 172.21.15.138:0/1295366275 wait complete. 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 Processor -- start 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.337+0000 7f6939212880 1 -- start start 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6939212880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6939212880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edd42d0780 con 0x55edd462cc00 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6934eb1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6934eb1640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44050/0 (socket says 172.21.15.138:44050) 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6934eb1640 1 -- 172.21.15.138:0/2824239615 learned_addr learned my addr 172.21.15.138:0/2824239615 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6934eb1640 1 -- 172.21.15.138:0/2824239615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edd431e4e0 con 0x55edd462cc00 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6934eb1640 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55edd4617380 tx=0x55edd45f0ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f691a67c640 1 -- 172.21.15.138:0/2824239615 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd4652840 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f691a67c640 1 -- 172.21.15.138:0/2824239615 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55edd508ca80 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f691a67c640 1 -- 172.21.15.138:0/2824239615 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55edd4652580 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55edd42b1380 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.338+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edd42b0ea0 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f691a67c640 1 -- 172.21.15.138:0/2824239615 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55edd42d7b80 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f691a67c640 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd520ac00 0x55edd4591700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f691a67c640 1 -- 172.21.15.138:0/2824239615 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55edd42bfdc0 con 0x55edd462cc00 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f6939212880 5 note: GC not initialized 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f6935eb3640 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd520ac00 0x55edd4591700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f6917e77640 20 reqs_thread_entry: start 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.339+0000 7f6939212880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6935eb3640 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd520ac00 0x55edd4591700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55edd46168a0 tx=0x55edd45f08a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6915e73640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:07 2024 2024-04-27T05:21:37.372 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6915651640 20 reqs_thread_entry: start 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720000 con 0x55edd50a5000 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720400 con 0x55edd462d000 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.340+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720800 con 0x55edd462d000 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 4.34 4:2e3daf19:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720c00 con 0x55edd462d000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 4.35 4:adc4a58e:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4721000 con 0x55edd50a5000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.2c 4:346a3ea0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4721400 con 0x55edd50a5000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 4.2e 4:74cdb6e4:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4721800 con 0x55edd462d800 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 4.3a 4:5decd619:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4721c00 con 0x55edd462d800 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 4.c 4:335b21e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4736000 con 0x55edd50a5000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4736400 con 0x55edd50a5000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.2f 4:f6a773d6:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4736800 con 0x55edd50a5000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.2 4:4123e525:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4736c00 con 0x55edd50a5000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 4.3e 4:7d356189:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4737000 con 0x55edd462d000 2024-04-27T05:21:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 4.16 4:6823909f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4737400 con 0x55edd462d800 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.341+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.342+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4737800 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.343+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(76 foo.buckets [call] v29'15 uv15 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.343+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4737c00 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.344+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(77 foo.buckets [call] v29'16 uv16 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.344+0000 7f6939212880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.344+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4737000 con 0x55edd50a5000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.344+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(78 foo [stat out=16b,read 0~357 out=357b] v0'0 uv11 ondisk = 0) v8 ==== 189+0+373 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.344+0000 7f6939212880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.344+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4736000 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.345+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) v8 ==== 155+0+58 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.345+0000 7f691c680640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.345+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94479958183168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4736400 con 0x55edd50a5000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.346+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 94479958183168] v29'109 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.346+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94479958173952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720400 con 0x55edd462d000 2024-04-27T05:21:37.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.347+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 94479958173952] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.347+0000 7f6939212880 20 remove_watcher() i=0 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.347+0000 7f6939212880 2 removed watcher, disabling cache 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.347+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94479958175104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720800 con 0x55edd50a5000 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.348+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 94479958175104] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.348+0000 7f6939212880 20 remove_watcher() i=1 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.348+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94479958176256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720c00 con 0x55edd462d800 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.349+0000 7f69356b2640 1 -- 172.21.15.138:0/3150561739 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 94479958176256] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642fc0 con 0x55edd462d800 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.349+0000 7f6939212880 20 remove_watcher() i=2 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.349+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94479958177408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4720000 con 0x55edd462d000 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.350+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 94479958177408] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4643200 con 0x55edd462d000 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.350+0000 7f6939212880 20 remove_watcher() i=3 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.350+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94479958178560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4721000 con 0x55edd50a5000 2024-04-27T05:21:37.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.351+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 94479958178560] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd50a5000 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.351+0000 7f6939212880 20 remove_watcher() i=4 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.351+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94479958179712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4721400 con 0x55edd462dc00 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.352+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 94479958179712] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.352+0000 7f6939212880 20 remove_watcher() i=5 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.352+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94479958180864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650000 con 0x55edd462dc00 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.353+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 94479958180864] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.353+0000 7f6939212880 20 remove_watcher() i=6 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.353+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94479958182016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55edd4650c00 con 0x55edd462dc00 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6935eb3640 1 -- 172.21.15.138:0/3150561739 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 94479958182016] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55edd4642900 con 0x55edd462dc00 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 20 remove_watcher() i=7 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55edd462dc00 msgr2=0x55edd4591180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55edd462dc00 0x55edd4591180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55edd50a5000 msgr2=0x55edd458fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55edd50a5000 0x55edd458fb80 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55edd462d000 msgr2=0x55edd4590c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55edd462d000 0x55edd4590c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55edd462d800 msgr2=0x55edd4590100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55edd462d800 0x55edd4590100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd462c000 msgr2=0x55edd458f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd462c000 0x55edd458f600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55edd43b9380 tx=0x55edd45f0120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 msgr2=0x55edd458eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 --2- 172.21.15.138:0/3150561739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd50a4c00 0x55edd458eb00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55edd51693e0 tx=0x55edd43bf9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 reap_dead start 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6934eb1640 1 -- 172.21.15.138:0/3150561739 reap_dead start 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 shutdown_connections 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 >> 172.21.15.138:0/3150561739 conn(0x55edd42ad400 msgr2=0x55edd43b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 shutdown_connections 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.354+0000 7f6939212880 1 -- 172.21.15.138:0/3150561739 wait complete. 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd520ac00 msgr2=0x55edd4591700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd520ac00 0x55edd4591700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55edd46168a0 tx=0x55edd45f08a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 msgr2=0x55edd4590680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55edd4617380 tx=0x55edd45f0ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 shutdown_connections 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd520ac00 0x55edd4591700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/2824239615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd462cc00 0x55edd4590680 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 >> 172.21.15.138:0/2824239615 conn(0x55edd4651800 msgr2=0x55edd43b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 shutdown_connections 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/2824239615 wait complete. 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd50a5c00 msgr2=0x55edd458f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd50a5c00 0x55edd458f080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55edd5198600 tx=0x55edd43beea0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 msgr2=0x55edd458e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55edd51692c0 tx=0x55edd43bec60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 shutdown_connections 2024-04-27T05:21:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55edd50a5c00 0x55edd458f080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 --2- 172.21.15.138:0/170029821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55edd42ad000 0x55edd458e580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 >> 172.21.15.138:0/170029821 conn(0x55edd42acc00 msgr2=0x55edd43b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 shutdown_connections 2024-04-27T05:21:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.355+0000 7f6939212880 1 -- 172.21.15.138:0/170029821 wait complete. 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "stats": { 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 3, 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "size_actual": 4096, 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb": 1, 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb_actual": 4, 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "last_stats_sync": "2024-04-27T05:21:37.345250Z", 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout: "last_stats_update": "2024-04-27T05:21:37.344000Z" 2024-04-27T05:21:37.382 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:37.383 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-04-27T05:21:37.345250Z', 'last_stats_update': '2024-04-27T05:21:37.344000Z'} 2024-04-27T05:21:37.387 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:37.387 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-27T05:21:37.387 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-27T05:21:37.388 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-04-27T05:21:37.466 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:37.467 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f2700c30880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f2700c30880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c92dbd6600 con 0x55c92d8ff400 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f2700c30880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c92d93e280 con 0x55c92d8ff000 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f26fcd90640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f26fc58f640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55c92d8ff400 0x55c92da06800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:60960/0 (socket says 172.21.15.138:60960) 2024-04-27T05:21:37.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.465+0000 7f26fc58f640 1 -- 172.21.15.138:0/4047365593 learned_addr learned my addr 172.21.15.138:0/4047365593 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:37.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.466+0000 7f26fcd90640 1 -- 172.21.15.138:0/4047365593 >> v1:172.21.15.114:6789/0 conn(0x55c92d8ff400 legacy=0x55c92da06800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.466+0000 7f26fcd90640 1 -- 172.21.15.138:0/4047365593 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d903380 con 0x55c92d8ff000 2024-04-27T05:21:37.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.467+0000 7f26fcd90640 1 --2- 172.21.15.138:0/4047365593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55c92da0ad50 tx=0x55c92da10de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9779b287135a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.467+0000 7f26fbd8e640 1 -- 172.21.15.138:0/4047365593 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d917ce0 con 0x55c92d8ff000 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.467+0000 7f26fbd8e640 1 -- 172.21.15.138:0/4047365593 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92dbd7b00 con 0x55c92d8ff000 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.467+0000 7f26fbd8e640 1 -- 172.21.15.138:0/4047365593 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d917e40 con 0x55c92d8ff000 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 -- 172.21.15.138:0/4047365593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 msgr2=0x55c92dbe0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 --2- 172.21.15.138:0/4047365593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55c92da0ad50 tx=0x55c92da10de0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 -- 172.21.15.138:0/4047365593 shutdown_connections 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 --2- 172.21.15.138:0/4047365593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 -- 172.21.15.138:0/4047365593 >> 172.21.15.138:0/4047365593 conn(0x55c92d8fec00 msgr2=0x55c92da06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 -- 172.21.15.138:0/4047365593 shutdown_connections 2024-04-27T05:21:37.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.468+0000 7f2700c30880 1 -- 172.21.15.138:0/4047365593 wait complete. 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f2700c30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f2700c30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92d93e780 con 0x55c92d8ff000 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fcd90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fcd90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44070/0 (socket says 172.21.15.138:44070) 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fcd90640 1 -- 172.21.15.138:0/1349045310 learned_addr learned my addr 172.21.15.138:0/1349045310 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fcd90640 1 -- 172.21.15.138:0/1349045310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d902ea0 con 0x55c92d8ff000 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fcd90640 1 --2- 172.21.15.138:0/1349045310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55c92e703560 tx=0x55c92da10cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ea3fceca5e6a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fb58d640 1 -- 172.21.15.138:0/1349045310 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d9174a0 con 0x55c92d8ff000 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fb58d640 1 -- 172.21.15.138:0/1349045310 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92dbd7e00 con 0x55c92d8ff000 2024-04-27T05:21:37.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.469+0000 7f26fb58d640 1 -- 172.21.15.138:0/1349045310 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d917ce0 con 0x55c92d8ff000 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- 172.21.15.138:0/1349045310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 msgr2=0x55c92dbe0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 --2- 172.21.15.138:0/1349045310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55c92e703560 tx=0x55c92da10cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- 172.21.15.138:0/1349045310 shutdown_connections 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 --2- 172.21.15.138:0/1349045310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- 172.21.15.138:0/1349045310 >> 172.21.15.138:0/1349045310 conn(0x55c92d8fec00 msgr2=0x55c92da07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- 172.21.15.138:0/1349045310 shutdown_connections 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- 172.21.15.138:0/1349045310 wait complete. 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.470+0000 7f2700c30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92d93e8c0 con 0x55c92d8ff000 2024-04-27T05:21:37.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26fcd90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26fcd90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44078/0 (socket says 172.21.15.138:44078) 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26fcd90640 1 -- 172.21.15.138:0/1303603835 learned_addr learned my addr 172.21.15.138:0/1303603835 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26fcd90640 1 -- 172.21.15.138:0/1303603835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d9036c0 con 0x55c92d8ff000 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26fcd90640 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55c92e7bade0 tx=0x55c92da10c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26f9d8a640 1 -- 172.21.15.138:0/1303603835 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d917e40 con 0x55c92d8ff000 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26f9d8a640 1 -- 172.21.15.138:0/1303603835 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92e6de000 con 0x55c92d8ff000 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c92d9031e0 con 0x55c92d8ff000 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f26f9d8a640 1 -- 172.21.15.138:0/1303603835 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d9174a0 con 0x55c92d8ff000 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.471+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c92d970ea0 con 0x55c92d8ff000 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26f9d8a640 1 -- 172.21.15.138:0/1303603835 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c92d928b00 con 0x55c92d8ff000 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f2700c30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f2700c30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92d922280 con 0x55c92e6f6c00 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26f9d8a640 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92e6f7c00 0x55c92dbe1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26f9d8a640 1 -- 172.21.15.138:0/1303603835 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c92d911a40 con 0x55c92d8ff000 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26fcd90640 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92e6f7c00 0x55c92dbe1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26fd591640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26fd591640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44084/0 (socket says 172.21.15.138:44084) 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26fd591640 1 -- 172.21.15.138:0/2060313064 learned_addr learned my addr 172.21.15.138:0/2060313064 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26fcd90640 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92e6f7c00 0x55c92dbe1080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55c92e7fa2a0 tx=0x55c92da10cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.472+0000 7f26fd591640 1 -- 172.21.15.138:0/2060313064 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d970d00 con 0x55c92e6f6c00 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f26fd591640 1 --2- 172.21.15.138:0/2060313064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55c92e7bb530 tx=0x55c92da11c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed7ecfe68b224f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f26f7585640 1 -- 172.21.15.138:0/2060313064 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d917e40 con 0x55c92e6f6c00 2024-04-27T05:21:37.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f26f7585640 1 -- 172.21.15.138:0/2060313064 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92e6de600 con 0x55c92e6f6c00 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f26f7585640 1 -- 172.21.15.138:0/2060313064 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d9174a0 con 0x55c92e6f6c00 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 -- 172.21.15.138:0/2060313064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 msgr2=0x55c92dbe0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 --2- 172.21.15.138:0/2060313064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55c92e7bb530 tx=0x55c92da11c20 comp rx=0 tx=0).stop 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 -- 172.21.15.138:0/2060313064 shutdown_connections 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 --2- 172.21.15.138:0/2060313064 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 -- 172.21.15.138:0/2060313064 >> 172.21.15.138:0/2060313064 conn(0x55c92d8ff400 msgr2=0x55c92da09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 -- 172.21.15.138:0/2060313064 shutdown_connections 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 -- 172.21.15.138:0/2060313064 wait complete. 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.473+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f2700c30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f2700c30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92d923180 con 0x55c92e6f6c00 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26fd591640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26fd591640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44090/0 (socket says 172.21.15.138:44090) 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 learned_addr learned my addr 172.21.15.138:0/2756258239 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d970b60 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26fd591640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c92e7035c0 tx=0x55c92da11980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26f5d82640 1 -- 172.21.15.138:0/2756258239 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d917e40 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f26f5d82640 1 -- 172.21.15.138:0/2756258239 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92e6de780 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.474+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c92d9709c0 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.475+0000 7f26f5d82640 1 -- 172.21.15.138:0/2756258239 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92d9174a0 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.475+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c92d970820 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.475+0000 7f26f5d82640 1 -- 172.21.15.138:0/2756258239 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c92d929600 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.475+0000 7f26f5d82640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dc7c000 0x55c92dbe1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.475+0000 7f26f5d82640 1 -- 172.21.15.138:0/2756258239 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c92d911c00 con 0x55c92e6f6c00 2024-04-27T05:21:37.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.475+0000 7f26fc58f640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dc7c000 0x55c92dbe1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c92e6f7000 0x55c92dbe1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7400 con 0x55c92e6f7000 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f26fd591640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c92e6f7000 0x55c92dbe1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f26fc58f640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dc7c000 0x55c92dbe1600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55c92e85af60 tx=0x55c92dc40120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f26fd591640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c92e6f7000 0x55c92dbe1b80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 20 realm 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7800 con 0x55c92e6f7000 2024-04-27T05:21:37.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.476+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7d000 con 0x55c92e6f7000 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7d400 con 0x55c92e6f7000 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c92dc7d800 0x55c92dbe2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7dc00 con 0x55c92dc7d800 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f26fcd90640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c92dc7d800 0x55c92dbe2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f26fcd90640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c92dc7d800 0x55c92dbe2100 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 20 searching for the correct realm 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.477+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7d000 con 0x55c92dc7d800 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fcd90640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c92dc7d000 0x55c92dbe2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7dc00 con 0x55c92dc7d000 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fc58f640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c92dc7d000 0x55c92dbe2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fc58f640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c92dc7d000 0x55c92dbe2c00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fc58f640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c92dc7dc00 0x55c92dbe3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.478+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.479+0000 7f26fd591640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c92dc7dc00 0x55c92dbe3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.479+0000 7f26fd591640 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c92dc7dc00 0x55c92dbe3180 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.479+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.479+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.479+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.479+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d000 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.480+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.481+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d000 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.482+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.483+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d000 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.484+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.485+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.485+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.485+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.485+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.485+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.485+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d000 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d800 2024-04-27T05:21:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92e6f7000 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7cc00 con 0x55c92dc7d000 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f2700c30880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f2700c30880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f2700c30880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.486+0000 7f2700c30880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7d400 con 0x55c92e6f7000 2024-04-27T05:21:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7400 con 0x55c92e6f7000 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7800 con 0x55c92dc7d800 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 zone default found 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 4 Realm: () 2024-04-27T05:21:37.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 zonegroup default 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.487+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0000 con 0x55c92dc7dc00 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0400 con 0x55c92dc7d800 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0800 con 0x55c92dc7d800 2024-04-27T05:21:37.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 started sync module instance, tier type = 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.488+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0c00 con 0x55c92dc7d000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.489+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.489+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94322544183552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca1000 con 0x55c92dc7d000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.489+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca1400 con 0x55c92e6f7000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.490+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94322544183552] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.490+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.490+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94322544184704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca1800 con 0x55c92e6f7000 2024-04-27T05:21:37.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.490+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca1c00 con 0x55c92dc7d800 2024-04-27T05:21:37.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.492+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.492+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94322544185856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0400 con 0x55c92dc7d800 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.492+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94322544184704] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.492+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0800 con 0x55c92dc7d000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.492+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94322544185856] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.493+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.493+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94322544187008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca1c00 con 0x55c92dc7d000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.493+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0000 con 0x55c92e6f7000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.494+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.494+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94322544187008] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.494+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94322544188160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7800 con 0x55c92e6f7000 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.494+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7400 con 0x55c92dc7dc00 2024-04-27T05:21:37.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.495+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.495+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94322544189312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dc7d400 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.495+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94322544188160] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.495+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e85cc00 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.496+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94322544189312] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.496+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.496+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94322544190464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e85c800 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.496+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0000 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94322544190464] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94322544191616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0c00 con 0x55c92dc7dc00 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=0 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=1 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=2 2024-04-27T05:21:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=3 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=4 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=5 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.497+0000 7f2700c30880 20 add_watcher() i=6 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.498+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94322544191616] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.498+0000 7f2700c30880 20 add_watcher() i=7 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.498+0000 7f2700c30880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.498+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca1000 con 0x55c92e6f7000 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.498+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.498+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94322544192768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dca0800 con 0x55c92e6f7000 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94322544192768] v29'110 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f26e3d5e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f2700c30880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f2700c30880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f2700c30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.499+0000 7f2700c30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92d922780 con 0x55c92dc7cc00 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26fc58f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26fc58f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44104/0 (socket says 172.21.15.138:44104) 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26fc58f640 1 -- 172.21.15.138:0/999640610 learned_addr learned my addr 172.21.15.138:0/999640610 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26fc58f640 1 -- 172.21.15.138:0/999640610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d970680 con 0x55c92dc7cc00 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26fc58f640 1 --2- 172.21.15.138:0/999640610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55c92dc5e930 tx=0x55c92dc40180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a8270318932f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26e355d640 1 -- 172.21.15.138:0/999640610 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92dca2580 con 0x55c92dc7cc00 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26e355d640 1 -- 172.21.15.138:0/999640610 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92e6de000 con 0x55c92dc7cc00 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.500+0000 7f26e355d640 1 -- 172.21.15.138:0/999640610 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92dca26e0 con 0x55c92dc7cc00 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- 172.21.15.138:0/999640610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 msgr2=0x55c92dbe2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 --2- 172.21.15.138:0/999640610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55c92dc5e930 tx=0x55c92dc40180 comp rx=0 tx=0).stop 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- 172.21.15.138:0/999640610 shutdown_connections 2024-04-27T05:21:37.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 --2- 172.21.15.138:0/999640610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- 172.21.15.138:0/999640610 >> 172.21.15.138:0/999640610 conn(0x55c92dc7d400 msgr2=0x55c92da09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- 172.21.15.138:0/999640610 shutdown_connections 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- 172.21.15.138:0/999640610 wait complete. 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 Processor -- start 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- start start 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.501+0000 7f2700c30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92d922500 con 0x55c92dc7cc00 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26fc58f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26fc58f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44106/0 (socket says 172.21.15.138:44106) 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26fc58f640 1 -- 172.21.15.138:0/4290913356 learned_addr learned my addr 172.21.15.138:0/4290913356 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26fc58f640 1 -- 172.21.15.138:0/4290913356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92d9704e0 con 0x55c92dc7cc00 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26fc58f640 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55c92dc5f830 tx=0x55c92dc40ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26e1d5a640 1 -- 172.21.15.138:0/4290913356 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92dca2840 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26e1d5a640 1 -- 172.21.15.138:0/4290913356 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c92e6dea80 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c92d903380 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f26e1d5a640 1 -- 172.21.15.138:0/4290913356 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c92dca2580 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.502+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c92d902ea0 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.503+0000 7f26e1d5a640 1 -- 172.21.15.138:0/4290913356 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c92d929b80 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.503+0000 7f26e1d5a640 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dd38000 0x55c92dbe3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.503+0000 7f26e1d5a640 1 -- 172.21.15.138:0/4290913356 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c92d911dc0 con 0x55c92dc7cc00 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.503+0000 7f2700c30880 5 note: GC not initialized 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.503+0000 7f26fd591640 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dd38000 0x55c92dbe3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.503+0000 7f26df555640 20 reqs_thread_entry: start 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f26fd591640 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dd38000 0x55c92dbe3700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c92e85a6c0 tx=0x55c92dc408a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f26dd551640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:07 2024 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f26dcd2f640 20 reqs_thread_entry: start 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e85c800 con 0x55c92e6f7000 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.504+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7800 con 0x55c92dc7d000 2024-04-27T05:21:37.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7400 con 0x55c92dc7d000 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:37.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.505+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd38400 con 0x55c92dc7d000 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 foo.buckets [call] v29'17 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd38800 con 0x55c92dc7d000 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.506+0000 7f2700c30880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.507+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39000 con 0x55c92dc7d000 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v29'18 uv18 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f2700c30880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f2700c30880 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f2700c30880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94322544195072 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39400 con 0x55c92dc7dc00 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== watch-notify(notify (1) cookie 94322544190464 notify 124554051594 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x55c92dc3fd40 con 0x55c92dc7dc00 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26f7585640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 94322544190464 notifier 4517 bl.length()=338 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26f7585640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26f7585640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(68 notify.6 [notify cookie 94322544195072 out=8b] v0'0 uv105 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26f7585640 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92e6f7800 con 0x55c92dc7dc00 2024-04-27T05:21:37.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.508+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.509+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== watch-notify(notify_complete (2) cookie 94322544195072 notify 124554051594 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c92dd01a00 con 0x55c92dc7dc00 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.509+0000 7f26e3d5e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.509+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94322544192768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39800 con 0x55c92e6f7000 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.510+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94322544192768] v29'111 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.510+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94322544183552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39c00 con 0x55c92dc7d000 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.511+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94322544183552] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.511+0000 7f2700c30880 20 remove_watcher() i=0 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.511+0000 7f2700c30880 2 removed watcher, disabling cache 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.511+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94322544184704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd38400 con 0x55c92e6f7000 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.512+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94322544184704] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.512+0000 7f2700c30880 20 remove_watcher() i=1 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.512+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94322544185856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd38800 con 0x55c92dc7d800 2024-04-27T05:21:37.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.513+0000 7f26fcd90640 1 -- 172.21.15.138:0/2756258239 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94322544185856] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96d80 con 0x55c92dc7d800 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.513+0000 7f2700c30880 20 remove_watcher() i=2 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.513+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94322544187008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39000 con 0x55c92dc7d000 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.514+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94322544187008] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc96fc0 con 0x55c92dc7d000 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.514+0000 7f2700c30880 20 remove_watcher() i=3 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.514+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94322544188160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd38c00 con 0x55c92e6f7000 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.515+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94322544188160] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92e6f7000 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.515+0000 7f2700c30880 20 remove_watcher() i=4 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.515+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94322544189312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39400 con 0x55c92dc7dc00 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.516+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94322544189312] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.516+0000 7f2700c30880 20 remove_watcher() i=5 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.516+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94322544190464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd39800 con 0x55c92dc7dc00 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.517+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94322544190464] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.517+0000 7f2700c30880 20 remove_watcher() i=6 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.517+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94322544191616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c92dd38400 con 0x55c92dc7dc00 2024-04-27T05:21:37.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f26fd591640 1 -- 172.21.15.138:0/2756258239 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94322544191616] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c92dc966c0 con 0x55c92dc7dc00 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 20 remove_watcher() i=7 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c92dc7dc00 msgr2=0x55c92dbe3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c92dc7dc00 0x55c92dbe3180 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c92e6f7000 msgr2=0x55c92dbe1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c92e6f7000 0x55c92dbe1b80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c92dc7d000 msgr2=0x55c92dbe2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c92dc7d000 0x55c92dbe2c00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c92dc7d800 msgr2=0x55c92dbe2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c92dc7d800 0x55c92dbe2100 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dc7c000 msgr2=0x55c92dbe1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dc7c000 0x55c92dbe1600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55c92e85af60 tx=0x55c92dc40120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 msgr2=0x55c92dbe0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f2700c30880 1 --2- 172.21.15.138:0/2756258239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92e6f6c00 0x55c92dbe0b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c92e7035c0 tx=0x55c92da11980 comp rx=0 tx=0).stop 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.518+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 reap_dead start 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f26fc58f640 1 -- 172.21.15.138:0/2756258239 reap_dead start 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 shutdown_connections 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 >> 172.21.15.138:0/2756258239 conn(0x55c92d8ff400 msgr2=0x55c92da09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 shutdown_connections 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/2756258239 wait complete. 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dd38000 msgr2=0x55c92dbe3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dd38000 0x55c92dbe3700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c92e85a6c0 tx=0x55c92dc408a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 msgr2=0x55c92dbe2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55c92dc5f830 tx=0x55c92dc40ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 shutdown_connections 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92dd38000 0x55c92dbe3700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/4290913356 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92dc7cc00 0x55c92dbe2680 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 >> 172.21.15.138:0/4290913356 conn(0x55c92dc7d400 msgr2=0x55c92da09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 shutdown_connections 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/4290913356 wait complete. 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92e6f7c00 msgr2=0x55c92dbe1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92e6f7c00 0x55c92dbe1080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55c92e7fa2a0 tx=0x55c92da10cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 msgr2=0x55c92dbe0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55c92e7bade0 tx=0x55c92da10c60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 shutdown_connections 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c92e6f7c00 0x55c92dbe1080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 --2- 172.21.15.138:0/1303603835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c92d8ff000 0x55c92dbe0580 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.519+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 >> 172.21.15.138:0/1303603835 conn(0x55c92d8fec00 msgr2=0x55c92da07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.520+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 shutdown_connections 2024-04-27T05:21:37.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.520+0000 7f2700c30880 1 -- 172.21.15.138:0/1303603835 wait complete. 2024-04-27T05:21:37.544 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T05:21:37.544 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T05:21:37.545 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-27T05:21:37.594 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fef36880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fef36880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b9cac54600 con 0x55b9ca97d400 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fef36880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b9ca9bc280 con 0x55b9ca97d000 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fb87e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fb87e640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58810/0 (socket says 172.21.15.138:58810) 2024-04-27T05:21:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fb07d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55b9ca97d400 0x55b9caa84800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42768/0 (socket says 172.21.15.138:42768) 2024-04-27T05:21:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fb87e640 1 -- 172.21.15.138:0/3400574757 learned_addr learned my addr 172.21.15.138:0/3400574757 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fb87e640 1 -- 172.21.15.138:0/3400574757 >> v1:172.21.15.114:6789/0 conn(0x55b9ca97d400 legacy=0x55b9caa84800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.593+0000 7fc7fb87e640 1 -- 172.21.15.138:0/3400574757 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca981380 con 0x55b9ca97d000 2024-04-27T05:21:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.595+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/3400574757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55b9caa88d50 tx=0x55b9caa8ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6378122fb2053e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.595+0000 7fc7fa87c640 1 -- 172.21.15.138:0/3400574757 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca995ce0 con 0x55b9ca97d000 2024-04-27T05:21:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.595+0000 7fc7fa87c640 1 -- 172.21.15.138:0/3400574757 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cac55b00 con 0x55b9ca97d000 2024-04-27T05:21:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.595+0000 7fc7fa87c640 1 -- 172.21.15.138:0/3400574757 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca995e40 con 0x55b9ca97d000 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 -- 172.21.15.138:0/3400574757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 msgr2=0x55b9cac5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 --2- 172.21.15.138:0/3400574757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55b9caa88d50 tx=0x55b9caa8ede0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 -- 172.21.15.138:0/3400574757 shutdown_connections 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 --2- 172.21.15.138:0/3400574757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 -- 172.21.15.138:0/3400574757 >> 172.21.15.138:0/3400574757 conn(0x55b9ca97cc00 msgr2=0x55b9caa84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 -- 172.21.15.138:0/3400574757 shutdown_connections 2024-04-27T05:21:37.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 -- 172.21.15.138:0/3400574757 wait complete. 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.596+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fef36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fef36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ca9bc780 con 0x55b9ca97d000 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fb87e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fb87e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58814/0 (socket says 172.21.15.138:58814) 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fb87e640 1 -- 172.21.15.138:0/3911849059 learned_addr learned my addr 172.21.15.138:0/3911849059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fb87e640 1 -- 172.21.15.138:0/3911849059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca980ea0 con 0x55b9ca97d000 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/3911849059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55b9cb781560 tx=0x55b9caa8ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40067868517d4e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fa07b640 1 -- 172.21.15.138:0/3911849059 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca9954a0 con 0x55b9ca97d000 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fa07b640 1 -- 172.21.15.138:0/3911849059 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cac55e00 con 0x55b9ca97d000 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.597+0000 7fc7fa07b640 1 -- 172.21.15.138:0/3911849059 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca995ce0 con 0x55b9ca97d000 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- 172.21.15.138:0/3911849059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 msgr2=0x55b9cac5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 --2- 172.21.15.138:0/3911849059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55b9cb781560 tx=0x55b9caa8ecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- 172.21.15.138:0/3911849059 shutdown_connections 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 --2- 172.21.15.138:0/3911849059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- 172.21.15.138:0/3911849059 >> 172.21.15.138:0/3911849059 conn(0x55b9ca97cc00 msgr2=0x55b9caa85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- 172.21.15.138:0/3911849059 shutdown_connections 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- 172.21.15.138:0/3911849059 wait complete. 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fef36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ca9bc8c0 con 0x55b9ca97d000 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fb87e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fb87e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58828/0 (socket says 172.21.15.138:58828) 2024-04-27T05:21:37.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.598+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1585964025 learned_addr learned my addr 172.21.15.138:0/1585964025 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1585964025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca9816c0 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55b9cb8392c0 tx=0x55b9caa8ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7f8878640 1 -- 172.21.15.138:0/1585964025 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca995e40 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7f8878640 1 -- 172.21.15.138:0/1585964025 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cb83a000 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b9ca9811e0 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7f8878640 1 -- 172.21.15.138:0/1585964025 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca9954a0 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9ca9eeea0 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.599+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fef36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fef36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ca9a0280 con 0x55b9cb774c00 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7f8878640 1 -- 172.21.15.138:0/1585964025 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b9ca9a6dc0 con 0x55b9ca97d000 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fc07f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7f8878640 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cb775c00 0x55b9cac5f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fc07f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58830/0 (socket says 172.21.15.138:58830) 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1365869972 learned_addr learned my addr 172.21.15.138:0/1365869972 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7f8878640 1 -- 172.21.15.138:0/1585964025 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b9ca98fa40 con 0x55b9ca97d000 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cb775c00 0x55b9cac5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1365869972 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca9eed00 con 0x55b9cb774c00 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/1365869972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55b9cb7815f0 tx=0x55b9caa8ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea5b0ebe570b8f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cb775c00 0x55b9cac5f080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55b9cb86e7b0 tx=0x55b9caa8ef60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7f6073640 1 -- 172.21.15.138:0/1365869972 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca995e40 con 0x55b9cb774c00 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7f6073640 1 -- 172.21.15.138:0/1365869972 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cb83a600 con 0x55b9cb774c00 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.600+0000 7fc7f6073640 1 -- 172.21.15.138:0/1365869972 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca9954a0 con 0x55b9cb774c00 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- 172.21.15.138:0/1365869972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 msgr2=0x55b9cac5eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1365869972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55b9cb7815f0 tx=0x55b9caa8ee40 comp rx=0 tx=0).stop 2024-04-27T05:21:37.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- 172.21.15.138:0/1365869972 shutdown_connections 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1365869972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- 172.21.15.138:0/1365869972 >> 172.21.15.138:0/1365869972 conn(0x55b9ca97d400 msgr2=0x55b9caa87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- 172.21.15.138:0/1365869972 shutdown_connections 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- 172.21.15.138:0/1365869972 wait complete. 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fef36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ca9a1180 con 0x55b9cb774c00 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fc07f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fc07f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58846/0 (socket says 172.21.15.138:58846) 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 learned_addr learned my addr 172.21.15.138:0/1247993912 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.601+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca9eeb60 con 0x55b9cb774c00 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55b9cb838b70 tx=0x55b9caa8f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7f4870640 1 -- 172.21.15.138:0/1247993912 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca995e40 con 0x55b9cb774c00 2024-04-27T05:21:37.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7f4870640 1 -- 172.21.15.138:0/1247993912 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cb83a780 con 0x55b9cb774c00 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b9ca9ee9c0 con 0x55b9cb774c00 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7f4870640 1 -- 172.21.15.138:0/1247993912 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9ca9954a0 con 0x55b9cb774c00 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9ca9ee820 con 0x55b9cb774c00 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.602+0000 7fc7f4870640 1 -- 172.21.15.138:0/1247993912 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b9ca9a6b00 con 0x55b9cb774c00 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7f4870640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cacfa000 0x55b9cac5f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7f4870640 1 -- 172.21.15.138:0/1247993912 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b9ca98fc00 con 0x55b9cb774c00 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cacfa000 0x55b9cac5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b9cb775000 0x55b9cac5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775400 con 0x55b9cb775000 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cacfa000 0x55b9cac5f600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55b9caa89380 tx=0x55b9cacbe120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b9cb775000 0x55b9cac5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.603+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b9cb775000 0x55b9cac5fb80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 realm 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775800 con 0x55b9cb775000 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfb000 con 0x55b9cb775000 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfb400 con 0x55b9cb775000 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b9cacfb800 0x55b9cac60100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfbc00 con 0x55b9cacfb800 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.604+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b9cacfb800 0x55b9cac60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b9cacfb800 0x55b9cac60100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fef36880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fef36880 20 searching for the correct realm 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfb000 con 0x55b9cacfb800 2024-04-27T05:21:37.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fb87e640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b9cacfb000 0x55b9cac60c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfbc00 con 0x55b9cacfb000 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.605+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b9cacfb000 0x55b9cac60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.606+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b9cacfb000 0x55b9cac60c00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.606+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.606+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b9cacfbc00 0x55b9cac61180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.606+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.606+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b9cacfbc00 0x55b9cac61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.606+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b9cacfbc00 0x55b9cac61180 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.607+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb000 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.608+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb000 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.609+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.610+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb000 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.611+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.612+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.612+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.612+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.612+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.612+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.612+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb000 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb800 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfbc00 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cb775000 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfac00 con 0x55b9cacfb000 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fef36880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fef36880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fef36880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.613+0000 7fc7fef36880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfb400 con 0x55b9cb775000 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775400 con 0x55b9cb775000 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775800 con 0x55b9cacfb800 2024-04-27T05:21:37.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 zone default found 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 4 Realm: () 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 zonegroup default 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.614+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20000 con 0x55b9cacfbc00 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20400 con 0x55b9cacfb800 2024-04-27T05:21:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20800 con 0x55b9cacfb800 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 started sync module instance, tier type = 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.615+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20c00 con 0x55b9cacfb000 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.616+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.616+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94256459245824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad21000 con 0x55b9cacfb000 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.616+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad21400 con 0x55b9cb775000 2024-04-27T05:21:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.617+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94256459245824] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.617+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.617+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94256459246976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad21800 con 0x55b9cb775000 2024-04-27T05:21:37.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.617+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad21c00 con 0x55b9cacfb800 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.618+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.618+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94256459248128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20400 con 0x55b9cacfb800 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.618+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20800 con 0x55b9cacfb000 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.618+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94256459246976] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.619+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94256459248128] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.619+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.619+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94256459249280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad21c00 con 0x55b9cacfb000 2024-04-27T05:21:37.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.619+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20000 con 0x55b9cb775000 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.620+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94256459249280] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.620+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.620+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94256459250432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775800 con 0x55b9cb775000 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.620+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775400 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.621+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94256459250432] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.622+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.622+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94256459251584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cacfb400 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.622+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb8db800 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94256459251584] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94256459252736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb8db400 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20000 con 0x55b9cacfbc00 2024-04-27T05:21:37.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94256459253888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20c00 con 0x55b9cacfbc00 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fef36880 20 add_watcher() i=0 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.623+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94256459252736] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=1 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=2 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=3 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=4 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=5 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=6 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94256459253888] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 20 add_watcher() i=7 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.624+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad21000 con 0x55b9cb775000 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.625+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.625+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94256459255040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cad20800 con 0x55b9cb775000 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94256459255040] v29'112 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7e284c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fef36880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fef36880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fef36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.626+0000 7fc7fef36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ca9a0780 con 0x55b9cacfac00 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7fb07d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7fb07d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58862/0 (socket says 172.21.15.138:58862) 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7fb07d640 1 -- 172.21.15.138:0/3141745604 learned_addr learned my addr 172.21.15.138:0/3141745604 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7fb07d640 1 -- 172.21.15.138:0/3141745604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca9ee680 con 0x55b9cacfac00 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/3141745604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55b9cacdab10 tx=0x55b9cacbe180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123012e95b93484e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7e204b640 1 -- 172.21.15.138:0/3141745604 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9cad22580 con 0x55b9cacfac00 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7e204b640 1 -- 172.21.15.138:0/3141745604 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cb83a000 con 0x55b9cacfac00 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.627+0000 7fc7e204b640 1 -- 172.21.15.138:0/3141745604 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9cad226e0 con 0x55b9cacfac00 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- 172.21.15.138:0/3141745604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 msgr2=0x55b9cac60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 --2- 172.21.15.138:0/3141745604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55b9cacdab10 tx=0x55b9cacbe180 comp rx=0 tx=0).stop 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- 172.21.15.138:0/3141745604 shutdown_connections 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 --2- 172.21.15.138:0/3141745604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- 172.21.15.138:0/3141745604 >> 172.21.15.138:0/3141745604 conn(0x55b9cacfb400 msgr2=0x55b9caa87800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- 172.21.15.138:0/3141745604 shutdown_connections 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- 172.21.15.138:0/3141745604 wait complete. 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 Processor -- start 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- start start 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.628+0000 7fc7fef36880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ca9a0500 con 0x55b9cacfac00 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fb07d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fb07d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58876/0 (socket says 172.21.15.138:58876) 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fb07d640 1 -- 172.21.15.138:0/163234576 learned_addr learned my addr 172.21.15.138:0/163234576 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fb07d640 1 -- 172.21.15.138:0/163234576 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ca9ee4e0 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fb07d640 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55b9cacdb800 tx=0x55b9cacbeae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7e0848640 1 -- 172.21.15.138:0/163234576 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9cad22840 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7e0848640 1 -- 172.21.15.138:0/163234576 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9cb83aa80 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b9ca981380 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7e0848640 1 -- 172.21.15.138:0/163234576 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b9cad22580 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.629+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9ca980ea0 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7e0848640 1 -- 172.21.15.138:0/163234576 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b9ca9a78c0 con 0x55b9cacfac00 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7e0848640 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cadb6000 0x55b9cac61700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7e0848640 1 -- 172.21.15.138:0/163234576 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55b9ca98fdc0 con 0x55b9cacfac00 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7fef36880 5 note: GC not initialized 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cadb6000 0x55b9cac61700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7de043640 20 reqs_thread_entry: start 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7fef36880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7fc07f640 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cadb6000 0x55b9cac61700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b9cb8e8d80 tx=0x55b9cacbe8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.630+0000 7fc7fef36880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7dc03f640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:07 2024 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7db81d640 20 reqs_thread_entry: start 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb8db400 con 0x55b9cacfbc00 2024-04-27T05:21:37.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775800 con 0x55b9cacfbc00 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.631+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cb775400 con 0x55b9cacfbc00 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.632+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v29'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.632+0000 7fc7fef36880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T05:21:37.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.632+0000 7fc7fef36880 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.632+0000 7fc7fef36880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.632+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94256459257344 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb6400 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== watch-notify(notify (1) cookie 94256459251584 notify 124554051595 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55b9cacbdd40 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7f6073640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051595 cookie 94256459251584 notifier 4525 bl.length()=522 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7f6073640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7f6073640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(65 notify.5 [notify cookie 94256459257344 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7f6073640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae06000 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== watch-notify(notify_complete (2) cookie 94256459257344 notify 124554051595 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b9cad7da00 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.633+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb6800 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.634+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v29'3 uv3 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.634+0000 7fc7fef36880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:21:37.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.634+0000 7fc7fef36880 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.634+0000 7fc7fef36880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.634+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94256459257344 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb6c00 con 0x55b9cacfb800 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94256459248128 notify 124554051587 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55b9cb88ad00 con 0x55b9cacfb800 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7f6073640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051587 cookie 94256459248128 notifier 4525 bl.length()=177 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7f6073640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7f6073640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7f6073640 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae06400 con 0x55b9cacfb800 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [notify cookie 94256459257344 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.635+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94256459257344 notify 124554051587 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b9cb88aea0 con 0x55b9cacfb800 2024-04-27T05:21:37.658 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "fud", 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Fud", 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "", 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 1, 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "fud", 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T05:21:37.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:37.660 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:37.661 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:37.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.636+0000 7fc7e284c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.636+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94256459255040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb7000 con 0x55b9cb775000 2024-04-27T05:21:37.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.637+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94256459255040] v29'113 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.637+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94256459245824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb7400 con 0x55b9cacfb000 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.638+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94256459245824] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.638+0000 7fc7fef36880 20 remove_watcher() i=0 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.638+0000 7fc7fef36880 2 removed watcher, disabling cache 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.638+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94256459246976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb7800 con 0x55b9cb775000 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.639+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94256459246976] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.639+0000 7fc7fef36880 20 remove_watcher() i=1 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.639+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94256459248128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cadb7c00 con 0x55b9cacfb800 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.640+0000 7fc7fb87e640 1 -- 172.21.15.138:0/1247993912 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94256459248128] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16d80 con 0x55b9cacfb800 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.640+0000 7fc7fef36880 20 remove_watcher() i=2 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.640+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94256459249280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae06000 con 0x55b9cacfb000 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.640+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94256459249280] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad16fc0 con 0x55b9cacfb000 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.640+0000 7fc7fef36880 20 remove_watcher() i=3 2024-04-27T05:21:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.641+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94256459250432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae06800 con 0x55b9cb775000 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.641+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94256459250432] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cb775000 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.641+0000 7fc7fef36880 20 remove_watcher() i=4 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.641+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94256459251584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae06c00 con 0x55b9cacfbc00 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.642+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94256459251584] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.642+0000 7fc7fef36880 20 remove_watcher() i=5 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.642+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94256459252736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae07000 con 0x55b9cacfbc00 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.643+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94256459252736] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.643+0000 7fc7fef36880 20 remove_watcher() i=6 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.643+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94256459253888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b9cae07400 con 0x55b9cacfbc00 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fc07f640 1 -- 172.21.15.138:0/1247993912 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94256459253888] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b9cad166c0 con 0x55b9cacfbc00 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 20 remove_watcher() i=7 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b9cacfbc00 msgr2=0x55b9cac61180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b9cacfbc00 0x55b9cac61180 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b9cb775000 msgr2=0x55b9cac5fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b9cb775000 0x55b9cac5fb80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b9cacfb000 msgr2=0x55b9cac60c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b9cacfb000 0x55b9cac60c00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b9cacfb800 msgr2=0x55b9cac60100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b9cacfb800 0x55b9cac60100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cacfa000 msgr2=0x55b9cac5f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cacfa000 0x55b9cac5f600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55b9caa89380 tx=0x55b9cacbe120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 msgr2=0x55b9cac5eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1247993912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cb774c00 0x55b9cac5eb00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55b9cb838b70 tx=0x55b9caa8f9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.644+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 reap_dead start 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 shutdown_connections 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 >> 172.21.15.138:0/1247993912 conn(0x55b9ca97d400 msgr2=0x55b9caa87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fb07d640 1 -- 172.21.15.138:0/1247993912 reap_dead start 2024-04-27T05:21:37.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 shutdown_connections 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1247993912 wait complete. 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cadb6000 msgr2=0x55b9cac61700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cadb6000 0x55b9cac61700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b9cb8e8d80 tx=0x55b9cacbe8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 msgr2=0x55b9cac60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55b9cacdb800 tx=0x55b9cacbeae0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 shutdown_connections 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cadb6000 0x55b9cac61700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/163234576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9cacfac00 0x55b9cac60680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 >> 172.21.15.138:0/163234576 conn(0x55b9cacfb400 msgr2=0x55b9caa87800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 shutdown_connections 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/163234576 wait complete. 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cb775c00 msgr2=0x55b9cac5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cb775c00 0x55b9cac5f080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55b9cb86e7b0 tx=0x55b9caa8ef60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 msgr2=0x55b9cac5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55b9cb8392c0 tx=0x55b9caa8ec60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 shutdown_connections 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b9cb775c00 0x55b9cac5f080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 --2- 172.21.15.138:0/1585964025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b9ca97d000 0x55b9cac5e580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.645+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 >> 172.21.15.138:0/1585964025 conn(0x55b9ca97cc00 msgr2=0x55b9caa85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.646+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 shutdown_connections 2024-04-27T05:21:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.646+0000 7fc7fef36880 1 -- 172.21.15.138:0/1585964025 wait complete. 2024-04-27T05:21:37.672 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:37.677 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T05:21:37.680 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:37.680 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-04-27T05:21:37.680 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-04-27T05:21:37.681 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-04-27T05:21:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.720+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.720+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.720+0000 7ffa16d26880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55caed505400 0x55caed7e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.720+0000 7ffa16d26880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55caed7dc600 con 0x55caed505000 2024-04-27T05:21:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.720+0000 7ffa16d26880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55caed544280 con 0x55caed505400 2024-04-27T05:21:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.720+0000 7ffa12dc2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55caed505400 0x55caed7e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.721+0000 7ffa135c3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55caed505000 0x55caed60c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42774/0 (socket says 172.21.15.138:42774) 2024-04-27T05:21:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.721+0000 7ffa135c3640 1 -- 172.21.15.138:0/4100295482 learned_addr learned my addr 172.21.15.138:0/4100295482 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.721+0000 7ffa12dc2640 1 -- 172.21.15.138:0/4100295482 >> v1:172.21.15.114:6789/0 conn(0x55caed505000 legacy=0x55caed60c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.721+0000 7ffa12dc2640 1 -- 172.21.15.138:0/4100295482 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed509380 con 0x55caed505400 2024-04-27T05:21:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.721+0000 7ffa125c1640 1 -- 172.21.15.138:0/4100295482 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245279414 0 0) 0x55caed5089c0 con 0x55caed505000 2024-04-27T05:21:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.722+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/4100295482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505400 0x55caed7e6580 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55caed610c00 tx=0x55caed616de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0071985c42a174 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa125c1640 1 -- 172.21.15.138:0/4100295482 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51dce0 con 0x55caed505400 2024-04-27T05:21:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa125c1640 1 -- 172.21.15.138:0/4100295482 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caed7ddb00 con 0x55caed505400 2024-04-27T05:21:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa125c1640 1 -- 172.21.15.138:0/4100295482 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51de40 con 0x55caed505400 2024-04-27T05:21:37.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 -- 172.21.15.138:0/4100295482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505400 msgr2=0x55caed7e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 --2- 172.21.15.138:0/4100295482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505400 0x55caed7e6580 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55caed610c00 tx=0x55caed616de0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 -- 172.21.15.138:0/4100295482 shutdown_connections 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 --2- 172.21.15.138:0/4100295482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505400 0x55caed7e6580 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 -- 172.21.15.138:0/4100295482 >> 172.21.15.138:0/4100295482 conn(0x55caed504c00 msgr2=0x55caed60c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 -- 172.21.15.138:0/4100295482 shutdown_connections 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.723+0000 7ffa16d26880 1 -- 172.21.15.138:0/4100295482 wait complete. 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa16d26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa16d26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caed544780 con 0x55caed505000 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa135c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa135c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58900/0 (socket says 172.21.15.138:58900) 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa135c3640 1 -- 172.21.15.138:0/3086726922 learned_addr learned my addr 172.21.15.138:0/3086726922 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.724+0000 7ffa135c3640 1 -- 172.21.15.138:0/3086726922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed508ea0 con 0x55caed505000 2024-04-27T05:21:37.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa135c3640 1 --2- 172.21.15.138:0/3086726922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55caee3b4c90 tx=0x55caed616cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add794ff9194dec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa11dc0640 1 -- 172.21.15.138:0/3086726922 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51d4a0 con 0x55caed505000 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa11dc0640 1 -- 172.21.15.138:0/3086726922 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caed7dde00 con 0x55caed505000 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa11dc0640 1 -- 172.21.15.138:0/3086726922 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51dce0 con 0x55caed505000 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 -- 172.21.15.138:0/3086726922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 msgr2=0x55caed7e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 --2- 172.21.15.138:0/3086726922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55caee3b4c90 tx=0x55caed616cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 -- 172.21.15.138:0/3086726922 shutdown_connections 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 --2- 172.21.15.138:0/3086726922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 -- 172.21.15.138:0/3086726922 >> 172.21.15.138:0/3086726922 conn(0x55caed504c00 msgr2=0x55caed60d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 -- 172.21.15.138:0/3086726922 shutdown_connections 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 -- 172.21.15.138:0/3086726922 wait complete. 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.725+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa16d26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa16d26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caed5448c0 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa135c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa135c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58908/0 (socket says 172.21.15.138:58908) 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa135c3640 1 -- 172.21.15.138:0/2489099413 learned_addr learned my addr 172.21.15.138:0/2489099413 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa135c3640 1 -- 172.21.15.138:0/2489099413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed5096c0 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa135c3640 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55caee3b5350 tx=0x55caed616c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa105bd640 1 -- 172.21.15.138:0/2489099413 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51de40 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa105bd640 1 -- 172.21.15.138:0/2489099413 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caee3c2000 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55caed5091e0 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa105bd640 1 -- 172.21.15.138:0/2489099413 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51d4a0 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.726+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caed576ea0 con 0x55caed505000 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa16d26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa16d26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caed544c80 con 0x55caee2f9800 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa105bd640 1 -- 172.21.15.138:0/2489099413 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55caed52f600 con 0x55caed505000 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa13dc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa13dc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58914/0 (socket says 172.21.15.138:58914) 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa13dc4640 1 -- 172.21.15.138:0/4116196780 learned_addr learned my addr 172.21.15.138:0/4116196780 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa105bd640 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee456000 0x55caed7e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa105bd640 1 -- 172.21.15.138:0/2489099413 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55caed517a40 con 0x55caed505000 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa13dc4640 1 -- 172.21.15.138:0/4116196780 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed576d00 con 0x55caee2f9800 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa135c3640 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee456000 0x55caed7e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.727+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/4116196780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55caee405440 tx=0x55caed617a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd0af2093f456fa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa0ddb8640 1 -- 172.21.15.138:0/4116196780 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51de40 con 0x55caee2f9800 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa0ddb8640 1 -- 172.21.15.138:0/4116196780 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caee3c2300 con 0x55caee2f9800 2024-04-27T05:21:37.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa135c3640 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee456000 0x55caed7e7080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55caee404450 tx=0x55caed6174a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa0ddb8640 1 -- 172.21.15.138:0/4116196780 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51d4a0 con 0x55caee2f9800 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 -- 172.21.15.138:0/4116196780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 msgr2=0x55caed7e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 --2- 172.21.15.138:0/4116196780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55caee405440 tx=0x55caed617a40 comp rx=0 tx=0).stop 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 -- 172.21.15.138:0/4116196780 shutdown_connections 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 --2- 172.21.15.138:0/4116196780 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 -- 172.21.15.138:0/4116196780 >> 172.21.15.138:0/4116196780 conn(0x55caed505400 msgr2=0x55caed60f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 -- 172.21.15.138:0/4116196780 shutdown_connections 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 -- 172.21.15.138:0/4116196780 wait complete. 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.728+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa16d26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa16d26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caed528280 con 0x55caee2f9800 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa13dc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa13dc4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58924/0 (socket says 172.21.15.138:58924) 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 learned_addr learned my addr 172.21.15.138:0/2320083667 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed576b60 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55caee3b5a70 tx=0x55caed617aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa0c5b5640 1 -- 172.21.15.138:0/2320083667 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51de40 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa0c5b5640 1 -- 172.21.15.138:0/2320083667 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caee3c2480 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55caed5769c0 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa0c5b5640 1 -- 172.21.15.138:0/2320083667 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caed51d4a0 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.729+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caed576820 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa0c5b5640 1 -- 172.21.15.138:0/2320083667 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55caed52f8c0 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa0c5b5640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caed888000 0x55caed7e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa0c5b5640 1 -- 172.21.15.138:0/2320083667 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55caed517c00 con 0x55caee2f9800 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caed888000 0x55caed7e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55caee2f9c00 0x55caed7e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888400 con 0x55caee2f9c00 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.730+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55caee2f9c00 0x55caed7e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caed888000 0x55caed7e7600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55caed610c00 tx=0x55caed852120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55caee2f9c00 0x55caed7e7b80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 realm 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888800 con 0x55caee2f9c00 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888c00 con 0x55caee2f9c00 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.731+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889000 con 0x55caee2f9c00 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55caed889400 0x55caed7e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888400 con 0x55caed889400 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa135c3640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55caed889400 0x55caed7e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa135c3640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55caed889400 0x55caed7e8100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 20 searching for the correct realm 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.732+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888800 con 0x55caed889400 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa135c3640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55caed888800 0x55caed7e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888400 con 0x55caed888800 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55caed888800 0x55caed7e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55caed888800 0x55caed7e8c00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55caed888400 0x55caed7e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.733+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.734+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55caed888400 0x55caed7e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.734+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55caed888400 0x55caed7e9180 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.734+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.734+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.734+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.734+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888800 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.735+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888800 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.736+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.737+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.738+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888800 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.739+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888800 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed889400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.740+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caee2f9c00 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2f8800 con 0x55caed888800 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888c00 con 0x55caee2f9c00 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889000 con 0x55caee2f9c00 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.741+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889800 con 0x55caed889400 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 zone default found 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 4 Realm: () 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 zonegroup default 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889c00 con 0x55caed888400 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8000 con 0x55caed889400 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.742+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8400 con 0x55caed889400 2024-04-27T05:21:37.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.743+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.743+0000 7ffa16d26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.743+0000 7ffa16d26880 20 started sync module instance, tier type = 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.743+0000 7ffa16d26880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.743+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8800 con 0x55caed888800 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.744+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.744+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94330056206592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8c00 con 0x55caed888800 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.744+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e9000 con 0x55caee2f9c00 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.744+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94330056206592] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.745+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.745+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94330056207744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e9400 con 0x55caee2f9c00 2024-04-27T05:21:37.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.745+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e9800 con 0x55caed889400 2024-04-27T05:21:37.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.746+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94330056207744] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.746+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.746+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94330056208896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8000 con 0x55caed889400 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.746+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8400 con 0x55caed888800 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.747+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94330056208896] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.747+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.747+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94330056210048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e9800 con 0x55caed888800 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.747+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889800 con 0x55caee2f9c00 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.748+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.748+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94330056210048] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.748+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94330056211200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed888c00 con 0x55caee2f9c00 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.748+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889000 con 0x55caed888400 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.749+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.749+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94330056212352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed889c00 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.749+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee457800 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.749+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94330056211200] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.750+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94330056212352] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.750+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.750+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94330056213504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8800 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.750+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8c00 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94330056213504] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94330056214656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e8400 con 0x55caed888400 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=0 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=1 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=2 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=3 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=4 2024-04-27T05:21:37.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=5 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.751+0000 7ffa16d26880 20 add_watcher() i=6 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.752+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94330056214656] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.752+0000 7ffa16d26880 20 add_watcher() i=7 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.752+0000 7ffa16d26880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.752+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e9800 con 0x55caee2f9c00 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.752+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.752+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94330056215808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee2e9000 con 0x55caee2f9c00 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.753+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94330056215808] v29'114 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.753+0000 7ff9fa591640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.753+0000 7ffa16d26880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.753+0000 7ffa16d26880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.753+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa16d26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa16d26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caed5288c0 con 0x55caee2e9c00 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa12dc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa12dc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58940/0 (socket says 172.21.15.138:58940) 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2602231641 learned_addr learned my addr 172.21.15.138:0/2602231641 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2602231641 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed576680 con 0x55caee2e9c00 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/2602231641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55caed86cc30 tx=0x55caed8520c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca09d4086e5b03b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ff9f9d90640 1 -- 172.21.15.138:0/2602231641 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caee2ea580 con 0x55caee2e9c00 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ff9f9d90640 1 -- 172.21.15.138:0/2602231641 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caee3c2000 con 0x55caee2e9c00 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.754+0000 7ff9f9d90640 1 -- 172.21.15.138:0/2602231641 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caee2ea6e0 con 0x55caee2e9c00 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 -- 172.21.15.138:0/2602231641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 msgr2=0x55caed7e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2602231641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55caed86cc30 tx=0x55caed8520c0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 -- 172.21.15.138:0/2602231641 shutdown_connections 2024-04-27T05:21:37.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2602231641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 -- 172.21.15.138:0/2602231641 >> 172.21.15.138:0/2602231641 conn(0x55caee2e9400 msgr2=0x55caed60f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 -- 172.21.15.138:0/2602231641 shutdown_connections 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 -- 172.21.15.138:0/2602231641 wait complete. 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 Processor -- start 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.755+0000 7ffa16d26880 1 -- start start 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa16d26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa16d26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caed528780 con 0x55caee2e9c00 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa12dc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa12dc2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58952/0 (socket says 172.21.15.138:58952) 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa12dc2640 1 -- 172.21.15.138:0/1076949624 learned_addr learned my addr 172.21.15.138:0/1076949624 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa12dc2640 1 -- 172.21.15.138:0/1076949624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caed508ea0 con 0x55caee2e9c00 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa12dc2640 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55caed86d7a0 tx=0x55caed852000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ff9f858d640 1 -- 172.21.15.138:0/1076949624 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caee2ea840 con 0x55caee2e9c00 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ff9f858d640 1 -- 172.21.15.138:0/1076949624 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55caee3c2780 con 0x55caee2e9c00 2024-04-27T05:21:37.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55caed5096c0 con 0x55caee2e9c00 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ff9f858d640 1 -- 172.21.15.138:0/1076949624 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55caee2ea580 con 0x55caee2e9c00 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.756+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55caed5091e0 con 0x55caee2e9c00 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ff9f858d640 1 -- 172.21.15.138:0/1076949624 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55caed52eb00 con 0x55caee2e9c00 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ff9f858d640 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee2e8800 0x55caed7e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ff9f858d640 1 -- 172.21.15.138:0/1076949624 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55caed517dc0 con 0x55caee2e9c00 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ffa16d26880 5 note: GC not initialized 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee2e8800 0x55caed7e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ff9f5d88640 20 reqs_thread_entry: start 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.757+0000 7ffa16d26880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ffa13dc4640 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee2e8800 0x55caed7e9700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55caee47aa20 tx=0x55caed852960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ffa16d26880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ff9f3d84640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ff9f3562640 20 reqs_thread_entry: start 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ffa16d26880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97e000 con 0x55caed888800 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.758+0000 7ffa16d26880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.759+0000 7ff9fa591640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.759+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94330056215808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97e400 con 0x55caee2f9c00 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.760+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94330056215808] v29'115 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.760+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94330056206592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97e800 con 0x55caed888800 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.761+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94330056206592] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.761+0000 7ffa16d26880 20 remove_watcher() i=0 2024-04-27T05:21:37.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.761+0000 7ffa16d26880 2 removed watcher, disabling cache 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.761+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94330056207744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97ec00 con 0x55caee2f9c00 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.762+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94330056207744] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.762+0000 7ffa16d26880 20 remove_watcher() i=1 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.762+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94330056208896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97f000 con 0x55caed889400 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.763+0000 7ffa135c3640 1 -- 172.21.15.138:0/2320083667 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94330056208896] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6fc0 con 0x55caed889400 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.763+0000 7ffa16d26880 20 remove_watcher() i=2 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.763+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94330056210048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97f400 con 0x55caed888800 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.764+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94330056210048] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a7200 con 0x55caed888800 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.764+0000 7ffa16d26880 20 remove_watcher() i=3 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.764+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94330056211200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97f800 con 0x55caee2f9c00 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.765+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94330056211200] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caee2f9c00 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.765+0000 7ffa16d26880 20 remove_watcher() i=4 2024-04-27T05:21:37.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.765+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94330056212352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caed97fc00 con 0x55caed888400 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.766+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94330056212352] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.766+0000 7ffa16d26880 20 remove_watcher() i=5 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.766+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94330056213504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee46c000 con 0x55caed888400 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.766+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94330056213504] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.766+0000 7ffa16d26880 20 remove_watcher() i=6 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.766+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94330056214656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55caee46c400 con 0x55caed888400 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa13dc4640 1 -- 172.21.15.138:0/2320083667 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94330056214656] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55caed8a6900 con 0x55caed888400 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 20 remove_watcher() i=7 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55caed888400 msgr2=0x55caed7e9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55caed888400 0x55caed7e9180 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55caee2f9c00 msgr2=0x55caed7e7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55caee2f9c00 0x55caed7e7b80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55caed888800 msgr2=0x55caed7e8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55caed888800 0x55caed7e8c00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55caed889400 msgr2=0x55caed7e8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55caed889400 0x55caed7e8100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caed888000 msgr2=0x55caed7e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caed888000 0x55caed7e7600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55caed610c00 tx=0x55caed852120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 msgr2=0x55caed7e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2320083667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2f9800 0x55caed7e6b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55caee3b5a70 tx=0x55caed617aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 reap_dead start 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa12dc2640 1 -- 172.21.15.138:0/2320083667 reap_dead start 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 shutdown_connections 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 >> 172.21.15.138:0/2320083667 conn(0x55caed505400 msgr2=0x55caed60f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 shutdown_connections 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/2320083667 wait complete. 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee2e8800 msgr2=0x55caed7e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee2e8800 0x55caed7e9700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55caee47aa20 tx=0x55caed852960 comp rx=0 tx=0).stop 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 msgr2=0x55caed7e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55caed86d7a0 tx=0x55caed852000 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 shutdown_connections 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee2e8800 0x55caed7e9700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 --2- 172.21.15.138:0/1076949624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caee2e9c00 0x55caed7e8680 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 >> 172.21.15.138:0/1076949624 conn(0x55caee2e9400 msgr2=0x55caed60f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 shutdown_connections 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.768+0000 7ffa16d26880 1 -- 172.21.15.138:0/1076949624 wait complete. 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee456000 msgr2=0x55caed7e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee456000 0x55caed7e7080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55caee404450 tx=0x55caed6174a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 msgr2=0x55caed7e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55caee3b5350 tx=0x55caed616c60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 shutdown_connections 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55caee456000 0x55caed7e7080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 --2- 172.21.15.138:0/2489099413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55caed505000 0x55caed7e6580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 >> 172.21.15.138:0/2489099413 conn(0x55caed504c00 msgr2=0x55caed60d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 shutdown_connections 2024-04-27T05:21:37.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.769+0000 7ffa16d26880 1 -- 172.21.15.138:0/2489099413 wait complete. 2024-04-27T05:21:37.791 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:37.791 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "bucket:myfoo", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": { 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "tag": "_FYwEPeTN9MwU6tJIQcJ0kGe", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": 2 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "mtime": "2024-04-27T05:21:37.508641Z", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "data": { 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": { 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "myfoo", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "marker": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_id": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "tenant": "", 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "explicit_placement": { 2024-04-27T05:21:37.792 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "", 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "", 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "" 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: "creation_time": "2024-04-27T05:21:36.328858Z", 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: "linked": false, 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: "has_bucket_info": false 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:37.793 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:37.793 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_FYwEPeTN9MwU6tJIQcJ0kGe', 'ver': 2}, 'mtime': '2024-04-27T05:21:37.508641Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'bucket_id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-04-27T05:21:36.328858Z', 'linked': False, 'has_bucket_info': False}} 2024-04-27T05:21:37.794 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1'] 2024-04-27T05:21:37.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1'] 2024-04-27T05:21:37.794 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:37.878 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:37.878 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.876+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.876+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7effa0e15880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5624a721d400 0x5624a74fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7effa0e15880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5624a74f4600 con 0x5624a721d000 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7effa0e15880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5624a725c280 con 0x5624a721d400 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7eff9ca0f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5624a721d400 0x5624a74fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7eff9d210640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5624a721d000 0x5624a7324800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42788/0 (socket says 172.21.15.138:42788) 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7eff9d210640 1 -- 172.21.15.138:0/491988075 learned_addr learned my addr 172.21.15.138:0/491988075 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7eff9ca0f640 1 -- 172.21.15.138:0/491988075 >> v1:172.21.15.114:6789/0 conn(0x5624a721d000 legacy=0x5624a7324800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.877+0000 7eff9ca0f640 1 -- 172.21.15.138:0/491988075 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a7221380 con 0x5624a721d400 2024-04-27T05:21:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/491988075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d400 0x5624a74fe580 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x5624a7328c00 tx=0x5624a732ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837ba25ac5d39012 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7eff9c20e640 1 -- 172.21.15.138:0/491988075 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a7235ce0 con 0x5624a721d400 2024-04-27T05:21:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7eff9c20e640 1 -- 172.21.15.138:0/491988075 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a74f5b00 con 0x5624a721d400 2024-04-27T05:21:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7eff9c20e640 1 -- 172.21.15.138:0/491988075 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a7235e40 con 0x5624a721d400 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7effa0e15880 1 -- 172.21.15.138:0/491988075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d400 msgr2=0x5624a74fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7effa0e15880 1 --2- 172.21.15.138:0/491988075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d400 0x5624a74fe580 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x5624a7328c00 tx=0x5624a732ede0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7effa0e15880 1 -- 172.21.15.138:0/491988075 shutdown_connections 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7effa0e15880 1 --2- 172.21.15.138:0/491988075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d400 0x5624a74fe580 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.879+0000 7effa0e15880 1 -- 172.21.15.138:0/491988075 >> 172.21.15.138:0/491988075 conn(0x5624a721cc00 msgr2=0x5624a7324000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7effa0e15880 1 -- 172.21.15.138:0/491988075 shutdown_connections 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7effa0e15880 1 -- 172.21.15.138:0/491988075 wait complete. 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7effa0e15880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7effa0e15880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a725c780 con 0x5624a721d000 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.880+0000 7eff9d210640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9d210640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58976/0 (socket says 172.21.15.138:58976) 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9d210640 1 -- 172.21.15.138:0/318820927 learned_addr learned my addr 172.21.15.138:0/318820927 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9d210640 1 -- 172.21.15.138:0/318820927 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a7220ea0 con 0x5624a721d000 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9d210640 1 --2- 172.21.15.138:0/318820927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5624a80d8cc0 tx=0x5624a732ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ce2619d916a468 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9ba0d640 1 -- 172.21.15.138:0/318820927 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a72354a0 con 0x5624a721d000 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9ba0d640 1 -- 172.21.15.138:0/318820927 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a74f5e00 con 0x5624a721d000 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7eff9ba0d640 1 -- 172.21.15.138:0/318820927 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a7235ce0 con 0x5624a721d000 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 -- 172.21.15.138:0/318820927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 msgr2=0x5624a74fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 --2- 172.21.15.138:0/318820927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5624a80d8cc0 tx=0x5624a732ecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 -- 172.21.15.138:0/318820927 shutdown_connections 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 --2- 172.21.15.138:0/318820927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 -- 172.21.15.138:0/318820927 >> 172.21.15.138:0/318820927 conn(0x5624a721cc00 msgr2=0x5624a7325000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 -- 172.21.15.138:0/318820927 shutdown_connections 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.881+0000 7effa0e15880 1 -- 172.21.15.138:0/318820927 wait complete. 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7effa0e15880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7effa0e15880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a725c8c0 con 0x5624a721d000 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7eff9d210640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7eff9d210640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58990/0 (socket says 172.21.15.138:58990) 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7eff9d210640 1 -- 172.21.15.138:0/1052970425 learned_addr learned my addr 172.21.15.138:0/1052970425 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7eff9d210640 1 -- 172.21.15.138:0/1052970425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a72216c0 con 0x5624a721d000 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.882+0000 7eff9d210640 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x5624a80d8ea0 tx=0x5624a732ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7eff9a20a640 1 -- 172.21.15.138:0/1052970425 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a7235e40 con 0x5624a721d000 2024-04-27T05:21:37.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7eff9a20a640 1 -- 172.21.15.138:0/1052970425 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a80ee000 con 0x5624a721d000 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624a72211e0 con 0x5624a721d000 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7eff9a20a640 1 -- 172.21.15.138:0/1052970425 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a72354a0 con 0x5624a721d000 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624a728eea0 con 0x5624a721d000 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7eff9a20a640 1 -- 172.21.15.138:0/1052970425 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5624a7247600 con 0x5624a721d000 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7effa0e15880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7effa0e15880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a7240280 con 0x5624a8015800 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.883+0000 7eff9a20a640 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a8174000 0x5624a74ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9a20a640 1 -- 172.21.15.138:0/1052970425 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5624a722fa40 con 0x5624a721d000 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9da11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9da11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:58996/0 (socket says 172.21.15.138:58996) 2024-04-27T05:21:37.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9da11640 1 -- 172.21.15.138:0/3576615900 learned_addr learned my addr 172.21.15.138:0/3576615900 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9d210640 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a8174000 0x5624a74ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9da11640 1 -- 172.21.15.138:0/3576615900 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a728ed00 con 0x5624a8015800 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9d210640 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a8174000 0x5624a74ff080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5624a8120450 tx=0x5624a732f4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff9da11640 1 --2- 172.21.15.138:0/3576615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5624a80d9f50 tx=0x5624a732ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be50a93649ea9da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff97a05640 1 -- 172.21.15.138:0/3576615900 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a7235e40 con 0x5624a8015800 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff97a05640 1 -- 172.21.15.138:0/3576615900 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a80ee300 con 0x5624a8015800 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7eff97a05640 1 -- 172.21.15.138:0/3576615900 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a72354a0 con 0x5624a8015800 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7effa0e15880 1 -- 172.21.15.138:0/3576615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 msgr2=0x5624a74feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.884+0000 7effa0e15880 1 --2- 172.21.15.138:0/3576615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5624a80d9f50 tx=0x5624a732ec00 comp rx=0 tx=0).stop 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 -- 172.21.15.138:0/3576615900 shutdown_connections 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 --2- 172.21.15.138:0/3576615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 -- 172.21.15.138:0/3576615900 >> 172.21.15.138:0/3576615900 conn(0x5624a721d400 msgr2=0x5624a7327000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 -- 172.21.15.138:0/3576615900 shutdown_connections 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 -- 172.21.15.138:0/3576615900 wait complete. 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7effa0e15880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a7241180 con 0x5624a8015800 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7eff9da11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7eff9da11640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59012/0 (socket says 172.21.15.138:59012) 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 learned_addr learned my addr 172.21.15.138:0/3049673478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a728eb60 con 0x5624a8015800 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.885+0000 7eff9da11640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x5624a80d9950 tx=0x5624a732f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7eff96202640 1 -- 172.21.15.138:0/3049673478 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a7235e40 con 0x5624a8015800 2024-04-27T05:21:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7eff96202640 1 -- 172.21.15.138:0/3049673478 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a80ee480 con 0x5624a8015800 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7eff96202640 1 -- 172.21.15.138:0/3049673478 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a72354a0 con 0x5624a8015800 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624a728e9c0 con 0x5624a8015800 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624a728e820 con 0x5624a8015800 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7eff96202640 1 -- 172.21.15.138:0/3049673478 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5624a72478c0 con 0x5624a8015800 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7eff96202640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75aa000 0x5624a74ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.886+0000 7eff96202640 1 -- 172.21.15.138:0/3049673478 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5624a722fc00 con 0x5624a8015800 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75aa000 0x5624a74ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5624a8015c00 0x5624a74ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aa400 con 0x5624a8015c00 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7eff9da11640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5624a8015c00 0x5624a74ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75aa000 0x5624a74ff600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5624a7328c00 tx=0x5624a7570120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7eff9da11640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5624a8015c00 0x5624a74ffb80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 20 realm 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aa800 con 0x5624a8015c00 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.887+0000 7effa0e15880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aac00 con 0x5624a8015c00 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ab000 con 0x5624a8015c00 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5624a75ab400 0x5624a7500100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aa400 con 0x5624a75ab400 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7eff9d210640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5624a75ab400 0x5624a7500100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.888+0000 7eff9d210640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5624a75ab400 0x5624a7500100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7effa0e15880 20 searching for the correct realm 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aa800 con 0x5624a75ab400 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7eff9d210640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5624a75aa800 0x5624a7500c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aa400 con 0x5624a75aa800 2024-04-27T05:21:37.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5624a75aa800 0x5624a7500c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.889+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5624a75aa800 0x5624a7500c00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5624a75aa400 0x5624a7501180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9da11640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5624a75aa400 0x5624a7501180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9da11640 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5624a75aa400 0x5624a7501180 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.890+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa800 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.891+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.892+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.892+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.892+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.892+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.892+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.892+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa800 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.893+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.894+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa800 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.895+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa800 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75ab400 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.896+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa400 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a8015c00 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8014800 con 0x5624a75aa800 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:37.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aac00 con 0x5624a8015c00 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.897+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ab000 con 0x5624a8015c00 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ab800 con 0x5624a75ab400 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 zone default found 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 4 Realm: () 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 zonegroup default 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75abc00 con 0x5624a75aa400 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc000 con 0x5624a75ab400 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.898+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc400 con 0x5624a75ab400 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.899+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.899+0000 7effa0e15880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.899+0000 7effa0e15880 20 started sync module instance, tier type = 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.899+0000 7effa0e15880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.899+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc800 con 0x5624a75aa800 2024-04-27T05:21:37.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.900+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.900+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94715425810688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ccc00 con 0x5624a75aa800 2024-04-27T05:21:37.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.900+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cd000 con 0x5624a8015c00 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.901+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94715425810688] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.901+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.901+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94715425811840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cd400 con 0x5624a8015c00 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.901+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cd800 con 0x5624a75ab400 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.902+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94715425811840] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.902+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.902+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94715425812992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc000 con 0x5624a75ab400 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.902+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc400 con 0x5624a75aa800 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.903+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94715425812992] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.903+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.903+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94715425814144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cd800 con 0x5624a75aa800 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.903+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ab800 con 0x5624a8015c00 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.904+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.904+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94715425814144] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.904+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94715425815296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75aac00 con 0x5624a8015c00 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.904+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ab000 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.905+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94715425815296] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.905+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.905+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94715425816448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75abc00 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.905+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a8175800 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.906+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.906+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94715425817600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc800 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.906+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75ccc00 con 0x5624a75aa400 2024-04-27T05:21:37.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.906+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94715425816448] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94715425817600] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94715425818752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cc400 con 0x5624a75aa400 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=0 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=1 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=2 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=3 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=4 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=5 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.908+0000 7effa0e15880 20 add_watcher() i=6 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.909+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94715425818752] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.909+0000 7effa0e15880 20 add_watcher() i=7 2024-04-27T05:21:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.909+0000 7effa0e15880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.909+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cd800 con 0x5624a8015c00 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.909+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.909+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94715425819904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a75cd000 con 0x5624a8015c00 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.910+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94715425819904] v29'116 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.910+0000 7eff841de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.910+0000 7effa0e15880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.910+0000 7effa0e15880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.910+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.910+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7effa0e15880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7effa0e15880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a7240780 con 0x5624a75cdc00 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7eff9ca0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7eff9ca0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59016/0 (socket says 172.21.15.138:59016) 2024-04-27T05:21:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7eff9ca0f640 1 -- 172.21.15.138:0/1384268650 learned_addr learned my addr 172.21.15.138:0/1384268650 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7eff9ca0f640 1 -- 172.21.15.138:0/1384268650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a728e680 con 0x5624a75cdc00 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/1384268650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5624a819a810 tx=0x5624a75700c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b8bdc86e8846b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.911+0000 7eff839dd640 1 -- 172.21.15.138:0/1384268650 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a75ce580 con 0x5624a75cdc00 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7eff839dd640 1 -- 172.21.15.138:0/1384268650 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a80ee000 con 0x5624a75cdc00 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7eff839dd640 1 -- 172.21.15.138:0/1384268650 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a75ce6e0 con 0x5624a75cdc00 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 -- 172.21.15.138:0/1384268650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 msgr2=0x5624a7500680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 --2- 172.21.15.138:0/1384268650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5624a819a810 tx=0x5624a75700c0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 -- 172.21.15.138:0/1384268650 shutdown_connections 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 --2- 172.21.15.138:0/1384268650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 -- 172.21.15.138:0/1384268650 >> 172.21.15.138:0/1384268650 conn(0x5624a75cd400 msgr2=0x5624a7327800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 -- 172.21.15.138:0/1384268650 shutdown_connections 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 -- 172.21.15.138:0/1384268650 wait complete. 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.912+0000 7effa0e15880 1 Processor -- start 2024-04-27T05:21:37.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7effa0e15880 1 -- start start 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7effa0e15880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7effa0e15880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624a7240500 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff9ca0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff9ca0f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59026/0 (socket says 172.21.15.138:59026) 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3073561301 learned_addr learned my addr 172.21.15.138:0/3073561301 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3073561301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624a728e4e0 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff9ca0f640 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x5624a758b740 tx=0x5624a7570000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff821da640 1 -- 172.21.15.138:0/3073561301 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a75ce840 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff821da640 1 -- 172.21.15.138:0/3073561301 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624a80ee780 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624a7220ea0 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7eff821da640 1 -- 172.21.15.138:0/3073561301 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5624a75ce580 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.913+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624a72216c0 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.914+0000 7eff821da640 1 -- 172.21.15.138:0/3073561301 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5624a7246b00 con 0x5624a75cdc00 2024-04-27T05:21:37.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.914+0000 7eff821da640 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75cc800 0x5624a7501700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.914+0000 7eff821da640 1 -- 172.21.15.138:0/3073561301 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5624a722fdc0 con 0x5624a75cdc00 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.914+0000 7eff9da11640 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75cc800 0x5624a7501700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.914+0000 7effa0e15880 5 note: GC not initialized 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7eff7f9d5640 20 reqs_thread_entry: start 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7eff9da11640 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75cc800 0x5624a7501700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5624a819ab40 tx=0x5624a7570960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7eff7d9d1640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7eff7d1af640 20 reqs_thread_entry: start 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768e000 con 0x5624a75aa400 2024-04-27T05:21:37.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.915+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768e400 con 0x5624a75aa800 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 20 rados->read ofs=0 len=0 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768e800 con 0x5624a75aa800 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.916+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768ec00 con 0x5624a75aa800 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.917+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 foo.buckets [call] v29'19 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.917+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768f400 con 0x5624a75aa800 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.918+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(66 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'4 uv4 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.918+0000 7effa0e15880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.918+0000 7effa0e15880 10 moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.918+0000 7effa0e15880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.918+0000 7effa0e15880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.918+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94715425822208 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768fc00 con 0x5624a75ab400 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94715425812992 notify 124554051588 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5624a8132ea0 con 0x5624a75ab400 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff97a05640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051588 cookie 94715425812992 notifier 4541 bl.length()=725 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(67 notify.2 [notify cookie 94715425822208 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff97a05640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff97a05640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:37.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff97a05640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff97a05640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76aa000 con 0x5624a75ab400 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== watch-notify(notify_complete (2) cookie 94715425822208 notify 124554051588 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5624a8133040 con 0x5624a75ab400 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7effa0e15880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_sources= new_sources= 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7effa0e15880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential sources added=[] removed=[] 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7effa0e15880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_dests= new_dests= 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7effa0e15880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential dests added=[] removed=[] 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.919+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76aa800 con 0x5624a75aa400 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.920+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 fud.buckets [call] v29'1 uv1 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.921+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76aac00 con 0x5624a75aa800 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v29'20 uv20 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7effa0e15880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T05:21:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7effa0e15880 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7effa0e15880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94715425822208 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76ab000 con 0x5624a75aa400 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== watch-notify(notify (1) cookie 94715425817600 notify 124554051596 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x5624a756fd40 con 0x5624a75aa400 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff97a05640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051596 cookie 94715425817600 notifier 4541 bl.length()=314 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff97a05640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff97a05640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(71 notify.6 [notify cookie 94715425822208 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff97a05640 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76aa400 con 0x5624a75aa400 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.922+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== watch-notify(notify_complete (2) cookie 94715425822208 notify 124554051596 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5624a7615860 con 0x5624a75aa400 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.923+0000 7eff841de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.923+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94715425819904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76ab400 con 0x5624a8015c00 2024-04-27T05:21:37.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.924+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94715425819904] v29'117 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.924+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94715425810688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76ab800 con 0x5624a75aa800 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.925+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94715425810688] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.925+0000 7effa0e15880 20 remove_watcher() i=0 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.925+0000 7effa0e15880 2 removed watcher, disabling cache 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.925+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94715425811840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76abc00 con 0x5624a8015c00 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.926+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94715425811840] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.926+0000 7effa0e15880 20 remove_watcher() i=1 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.926+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94715425812992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a76aa800 con 0x5624a75ab400 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.927+0000 7eff9d210640 1 -- 172.21.15.138:0/3049673478 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94715425812992] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75befc0 con 0x5624a75ab400 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.927+0000 7effa0e15880 20 remove_watcher() i=2 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.927+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94715425814144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768e000 con 0x5624a75aa800 2024-04-27T05:21:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.928+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94715425814144] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75bf200 con 0x5624a75aa800 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.928+0000 7effa0e15880 20 remove_watcher() i=3 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.928+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94715425815296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768f400 con 0x5624a8015c00 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.929+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94715425815296] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a8015c00 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.929+0000 7effa0e15880 20 remove_watcher() i=4 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.929+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94715425816448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768f800 con 0x5624a75aa400 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.930+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94715425816448] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.930+0000 7effa0e15880 20 remove_watcher() i=5 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.930+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94715425817600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768ec00 con 0x5624a75aa400 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.931+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94715425817600] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.931+0000 7effa0e15880 20 remove_watcher() i=6 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.931+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94715425818752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5624a768e800 con 0x5624a75aa400 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7eff9da11640 1 -- 172.21.15.138:0/3049673478 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94715425818752] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5624a75be900 con 0x5624a75aa400 2024-04-27T05:21:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 20 remove_watcher() i=7 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5624a75aa400 msgr2=0x5624a7501180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5624a75aa400 0x5624a7501180 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5624a8015c00 msgr2=0x5624a74ffb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5624a8015c00 0x5624a74ffb80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5624a75aa800 msgr2=0x5624a7500c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5624a75aa800 0x5624a7500c00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5624a75ab400 msgr2=0x5624a7500100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5624a75ab400 0x5624a7500100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75aa000 msgr2=0x5624a74ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75aa000 0x5624a74ff600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5624a7328c00 tx=0x5624a7570120 comp rx=0 tx=0).stop 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 msgr2=0x5624a74feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 --2- 172.21.15.138:0/3049673478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a8015800 0x5624a74feb00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x5624a80d9950 tx=0x5624a732f1a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 reap_dead start 2024-04-27T05:21:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7eff9ca0f640 1 -- 172.21.15.138:0/3049673478 reap_dead start 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 shutdown_connections 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 >> 172.21.15.138:0/3049673478 conn(0x5624a721d400 msgr2=0x5624a7327000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 shutdown_connections 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.932+0000 7effa0e15880 1 -- 172.21.15.138:0/3049673478 wait complete. 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75cc800 msgr2=0x5624a7501700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75cc800 0x5624a7501700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5624a819ab40 tx=0x5624a7570960 comp rx=0 tx=0).stop 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 msgr2=0x5624a7500680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x5624a758b740 tx=0x5624a7570000 comp rx=0 tx=0).stop 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 shutdown_connections 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a75cc800 0x5624a7501700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/3073561301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a75cdc00 0x5624a7500680 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 >> 172.21.15.138:0/3073561301 conn(0x5624a75cd400 msgr2=0x5624a7327800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 shutdown_connections 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/3073561301 wait complete. 2024-04-27T05:21:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a8174000 msgr2=0x5624a74ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a8174000 0x5624a74ff080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5624a8120450 tx=0x5624a732f4a0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 msgr2=0x5624a74fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x5624a80d8ea0 tx=0x5624a732ec60 comp rx=0 tx=0).stop 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 shutdown_connections 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5624a8174000 0x5624a74ff080 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 --2- 172.21.15.138:0/1052970425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5624a721d000 0x5624a74fe580 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 >> 172.21.15.138:0/1052970425 conn(0x5624a721cc00 msgr2=0x5624a7325000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 shutdown_connections 2024-04-27T05:21:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.933+0000 7effa0e15880 1 -- 172.21.15.138:0/1052970425 wait complete. 2024-04-27T05:21:37.958 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T05:21:37.959 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T05:21:37.959 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T05:21:38.001 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.001 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.999+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.999+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.999+0000 7f9e0e719880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.999+0000 7f9e0e719880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55eb49994600 con 0x55eb496bd400 2024-04-27T05:21:38.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:37.999+0000 7f9e0e719880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55eb496fc280 con 0x55eb496bd000 2024-04-27T05:21:38.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.000+0000 7f9e0cebe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.000+0000 7f9e0c6bd640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55eb496bd400 0x55eb497c4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42798/0 (socket says 172.21.15.138:42798) 2024-04-27T05:21:38.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.000+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/645950508 learned_addr learned my addr 172.21.15.138:0/645950508 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.000+0000 7f9e0cebe640 1 -- 172.21.15.138:0/645950508 >> v1:172.21.15.114:6789/0 conn(0x55eb496bd400 legacy=0x55eb497c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.000+0000 7f9e0cebe640 1 -- 172.21.15.138:0/645950508 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb496c1380 con 0x55eb496bd000 2024-04-27T05:21:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/645950508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55eb497c8d50 tx=0x55eb497cede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a30df4b147c643 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0bebc640 1 -- 172.21.15.138:0/645950508 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d5ce0 con 0x55eb496bd000 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0bebc640 1 -- 172.21.15.138:0/645950508 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb49995b00 con 0x55eb496bd000 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0bebc640 1 -- 172.21.15.138:0/645950508 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d5e40 con 0x55eb496bd000 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 -- 172.21.15.138:0/645950508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 msgr2=0x55eb4999e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 --2- 172.21.15.138:0/645950508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55eb497c8d50 tx=0x55eb497cede0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 -- 172.21.15.138:0/645950508 shutdown_connections 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 --2- 172.21.15.138:0/645950508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 -- 172.21.15.138:0/645950508 >> 172.21.15.138:0/645950508 conn(0x55eb496bcc00 msgr2=0x55eb497c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 -- 172.21.15.138:0/645950508 shutdown_connections 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.002+0000 7f9e0e719880 1 -- 172.21.15.138:0/645950508 wait complete. 2024-04-27T05:21:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0e719880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0e719880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb496fc780 con 0x55eb496bd000 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0cebe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0cebe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59052/0 (socket says 172.21.15.138:59052) 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2498236819 learned_addr learned my addr 172.21.15.138:0/2498236819 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.003+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2498236819 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb496c0ea0 con 0x55eb496bd000 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/2498236819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55eb4a4c1560 tx=0x55eb497cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255b53d9884f32fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0b6bb640 1 -- 172.21.15.138:0/2498236819 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d54a0 con 0x55eb496bd000 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0b6bb640 1 -- 172.21.15.138:0/2498236819 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb49995e00 con 0x55eb496bd000 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0b6bb640 1 -- 172.21.15.138:0/2498236819 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d5ce0 con 0x55eb496bd000 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- 172.21.15.138:0/2498236819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 msgr2=0x55eb4999e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2498236819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55eb4a4c1560 tx=0x55eb497cecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- 172.21.15.138:0/2498236819 shutdown_connections 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2498236819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- 172.21.15.138:0/2498236819 >> 172.21.15.138:0/2498236819 conn(0x55eb496bcc00 msgr2=0x55eb497c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- 172.21.15.138:0/2498236819 shutdown_connections 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- 172.21.15.138:0/2498236819 wait complete. 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.004+0000 7f9e0e719880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb496fc8c0 con 0x55eb496bd000 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0cebe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0cebe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59054/0 (socket says 172.21.15.138:59054) 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0cebe640 1 -- 172.21.15.138:0/967022096 learned_addr learned my addr 172.21.15.138:0/967022096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0cebe640 1 -- 172.21.15.138:0/967022096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb496c16c0 con 0x55eb496bd000 2024-04-27T05:21:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55eb4a578de0 tx=0x55eb497cec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e09eb8640 1 -- 172.21.15.138:0/967022096 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d5e40 con 0x55eb496bd000 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e09eb8640 1 -- 172.21.15.138:0/967022096 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb4a49c000 con 0x55eb496bd000 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb496c11e0 con 0x55eb496bd000 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e09eb8640 1 -- 172.21.15.138:0/967022096 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d54a0 con 0x55eb496bd000 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.005+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb4972eea0 con 0x55eb496bd000 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e09eb8640 1 -- 172.21.15.138:0/967022096 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55eb496e6b00 con 0x55eb496bd000 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0e719880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0e719880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb496e0280 con 0x55eb4a4b4c00 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0d6bf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e09eb8640 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a4b5c00 0x55eb4999f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0d6bf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59066/0 (socket says 172.21.15.138:59066) 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/1373895066 learned_addr learned my addr 172.21.15.138:0/1373895066 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e09eb8640 1 -- 172.21.15.138:0/967022096 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55eb496cfa40 con 0x55eb496bd000 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a4b5c00 0x55eb4999f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.006+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/1373895066 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb4972ed00 con 0x55eb4a4b4c00 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/1373895066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55eb4a5bd5c0 tx=0x55eb497cfc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0ffc2848ab6490 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a4b5c00 0x55eb4999f080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55eb4a5bc2a0 tx=0x55eb497cf920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e076b3640 1 -- 172.21.15.138:0/1373895066 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d5e40 con 0x55eb4a4b4c00 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e076b3640 1 -- 172.21.15.138:0/1373895066 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb4a49c600 con 0x55eb4a4b4c00 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e076b3640 1 -- 172.21.15.138:0/1373895066 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d54a0 con 0x55eb4a4b4c00 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 -- 172.21.15.138:0/1373895066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 msgr2=0x55eb4999eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 --2- 172.21.15.138:0/1373895066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55eb4a5bd5c0 tx=0x55eb497cfc20 comp rx=0 tx=0).stop 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 -- 172.21.15.138:0/1373895066 shutdown_connections 2024-04-27T05:21:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 --2- 172.21.15.138:0/1373895066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 -- 172.21.15.138:0/1373895066 >> 172.21.15.138:0/1373895066 conn(0x55eb496bd400 msgr2=0x55eb497c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 -- 172.21.15.138:0/1373895066 shutdown_connections 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 -- 172.21.15.138:0/1373895066 wait complete. 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.007+0000 7f9e0e719880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0e719880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb496e1180 con 0x55eb4a4b4c00 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0d6bf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0d6bf640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59076/0 (socket says 172.21.15.138:59076) 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 learned_addr learned my addr 172.21.15.138:0/443774368 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb4972eb60 con 0x55eb4a4b4c00 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55eb4a4c15c0 tx=0x55eb497cf9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e05eb0640 1 -- 172.21.15.138:0/443774368 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d5e40 con 0x55eb4a4b4c00 2024-04-27T05:21:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e05eb0640 1 -- 172.21.15.138:0/443774368 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb4a49c780 con 0x55eb4a4b4c00 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb4972e9c0 con 0x55eb4a4b4c00 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e05eb0640 1 -- 172.21.15.138:0/443774368 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb496d54a0 con 0x55eb4a4b4c00 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.008+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb4972e820 con 0x55eb4a4b4c00 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e05eb0640 1 -- 172.21.15.138:0/443774368 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55eb496e7600 con 0x55eb4a4b4c00 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e05eb0640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb49a3a000 0x55eb4999f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e05eb0640 1 -- 172.21.15.138:0/443774368 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55eb496cfc00 con 0x55eb4a4b4c00 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb49a3a000 0x55eb4999f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a4b5000 0x55eb4999fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5400 con 0x55eb4a4b5000 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.009+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a4b5000 0x55eb4999fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb49a3a000 0x55eb4999f600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55eb4a628a80 tx=0x55eb499fe120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a4b5000 0x55eb4999fb80 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 realm 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5800 con 0x55eb4a4b5000 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3b000 con 0x55eb4a4b5000 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.010+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3b400 con 0x55eb4a4b5000 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49a3b800 0x55eb499a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3bc00 con 0x55eb49a3b800 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49a3b800 0x55eb499a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49a3b800 0x55eb499a0100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 20 searching for the correct realm 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.011+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3b000 con 0x55eb49a3b800 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49a3b000 0x55eb499a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3bc00 con 0x55eb49a3b000 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49a3b000 0x55eb499a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49a3b000 0x55eb499a0c00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49a3bc00 0x55eb499a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.012+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.013+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49a3bc00 0x55eb499a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.013+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49a3bc00 0x55eb499a1180 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.013+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.013+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.013+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.013+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b000 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.014+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b000 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.015+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.016+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.017+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.018+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b000 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b800 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3bc00 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb4a4b5000 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.019+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3ac00 con 0x55eb49a3b000 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3b400 con 0x55eb4a4b5000 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5400 con 0x55eb4a4b5000 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5800 con 0x55eb49a3b800 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 zone default found 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 4 Realm: () 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 zonegroup default 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.020+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e000 con 0x55eb49a3bc00 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e400 con 0x55eb49a3b800 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e800 con 0x55eb49a3b800 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 started sync module instance, tier type = 2024-04-27T05:21:38.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.021+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5ec00 con 0x55eb49a3b000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.022+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.022+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94469040466176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5f000 con 0x55eb49a3b000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.022+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5f400 con 0x55eb4a4b5000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.023+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94469040466176] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.023+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.023+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94469040467328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5f800 con 0x55eb4a4b5000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.023+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5fc00 con 0x55eb49a3b800 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.024+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94469040467328] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.025+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.025+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94469040468480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e400 con 0x55eb49a3b800 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.025+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e800 con 0x55eb49a3b000 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.025+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94469040468480] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.026+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.026+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94469040469632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5fc00 con 0x55eb49a3b000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.026+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e000 con 0x55eb4a4b5000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.027+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94469040469632] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.027+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.027+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94469040470784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5800 con 0x55eb4a4b5000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.027+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5400 con 0x55eb49a3bc00 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.027+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94469040470784] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.028+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.028+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94469040471936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a3b400 con 0x55eb49a3bc00 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.028+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a61b800 con 0x55eb49a3bc00 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.029+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.029+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94469040473088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a61b400 con 0x55eb49a3bc00 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.029+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5ec00 con 0x55eb49a3bc00 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.029+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94469040471936] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94469040473088] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94469040474240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5f000 con 0x55eb49a3bc00 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=0 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=1 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=2 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=3 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=4 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=5 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=6 2024-04-27T05:21:38.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94469040474240] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 20 add_watcher() i=7 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.030+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e800 con 0x55eb4a4b5000 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.031+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.031+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94469040475392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5f400 con 0x55eb4a4b5000 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94469040475392] v29'118 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9df3e8c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0e719880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0e719880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0e719880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0e719880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb496e0780 con 0x55eb49a3ac00 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0c6bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0c6bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59084/0 (socket says 172.21.15.138:59084) 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.032+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/1676981389 learned_addr learned my addr 172.21.15.138:0/1676981389 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.033+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/1676981389 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb4972e680 con 0x55eb49a3ac00 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.033+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/1676981389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55eb49a1c930 tx=0x55eb499fe180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b40660bd4d80e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.033+0000 7f9df368b640 1 -- 172.21.15.138:0/1676981389 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb49a60580 con 0x55eb49a3ac00 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.033+0000 7f9df368b640 1 -- 172.21.15.138:0/1676981389 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb4a49c000 con 0x55eb49a3ac00 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.033+0000 7f9df368b640 1 -- 172.21.15.138:0/1676981389 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb49a606e0 con 0x55eb49a3ac00 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- 172.21.15.138:0/1676981389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 msgr2=0x55eb499a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 --2- 172.21.15.138:0/1676981389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55eb49a1c930 tx=0x55eb499fe180 comp rx=0 tx=0).stop 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- 172.21.15.138:0/1676981389 shutdown_connections 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 --2- 172.21.15.138:0/1676981389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- 172.21.15.138:0/1676981389 >> 172.21.15.138:0/1676981389 conn(0x55eb49a5f800 msgr2=0x55eb497c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- 172.21.15.138:0/1676981389 shutdown_connections 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- 172.21.15.138:0/1676981389 wait complete. 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 Processor -- start 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- start start 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0e719880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb496e0500 con 0x55eb49a3ac00 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0c6bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0c6bd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59086/0 (socket says 172.21.15.138:59086) 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.034+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 learned_addr learned my addr 172.21.15.138:0/2613031087 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb4972e4e0 con 0x55eb49a3ac00 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55eb49a1d860 tx=0x55eb499feae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9df1e88640 1 -- 172.21.15.138:0/2613031087 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb49a60840 con 0x55eb49a3ac00 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9df1e88640 1 -- 172.21.15.138:0/2613031087 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb4a49ca80 con 0x55eb49a3ac00 2024-04-27T05:21:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eb496c1380 con 0x55eb49a3ac00 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9df1e88640 1 -- 172.21.15.138:0/2613031087 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55eb49a60580 con 0x55eb49a3ac00 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.035+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb496c0ea0 con 0x55eb49a3ac00 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9df1e88640 1 -- 172.21.15.138:0/2613031087 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55eb496e7b80 con 0x55eb49a3ac00 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9df1e88640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a61b800 0x55eb499a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9df1e88640 1 -- 172.21.15.138:0/2613031087 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55eb496cfdc0 con 0x55eb49a3ac00 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a61b800 0x55eb499a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a61b400 0x55eb49afe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b02000 con 0x55eb4a61b400 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a61b400 0x55eb49afe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a61b800 0x55eb499a1700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55eb49a1d230 tx=0x55eb499fe8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.036+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a61b400 0x55eb49afe000 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.037+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'48 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb4a61b400 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.037+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49b02400 0x55eb49afe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.037+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b02800 con 0x55eb49b02400 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.037+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49b02400 0x55eb49afe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.038+0000 7f9e0c6bd640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49b02400 0x55eb49afe580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.039+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.039+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49b02c00 0x55eb49afeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.039+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03000 con 0x55eb49b02c00 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.039+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49b02c00 0x55eb49afeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.039+0000 7f9e0d6bf640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49b02c00 0x55eb49afeb00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.040+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.040+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03400 con 0x55eb4a61b400 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.041+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'119 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb4a61b400 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.041+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49b03800 0x55eb49aff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.041+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03c00 con 0x55eb49b03800 2024-04-27T05:21:38.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.041+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49b03800 0x55eb49aff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.041+0000 7f9e0cebe640 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49b03800 0x55eb49aff080 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.042+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'38 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.042+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e000 con 0x55eb49b02c00 2024-04-27T05:21:38.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.043+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'36 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.043+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e400 con 0x55eb4a61b400 2024-04-27T05:21:38.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.044+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb4a61b400 2024-04-27T05:21:38.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.044+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e800 con 0x55eb49b02400 2024-04-27T05:21:38.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.044+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'27 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.045+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2ec00 con 0x55eb49b03800 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.045+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.045+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f000 con 0x55eb49b02c00 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.046+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.046+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f400 con 0x55eb49b03800 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.047+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'39 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.047+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f800 con 0x55eb49b02400 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.048+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.048+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2fc00 con 0x55eb49b03800 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.049+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'20 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.049+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e400 con 0x55eb49b02400 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.050+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'44 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.050+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2ec00 con 0x55eb4a61b400 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.051+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'120 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb4a61b400 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.051+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f400 con 0x55eb49b02c00 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.052+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'32 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.052+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b02000 con 0x55eb49b03800 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.053+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'40 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.053+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03400 con 0x55eb49b02c00 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.054+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.054+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03c00 con 0x55eb4a61b400 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.055+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb4a61b400 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.055+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5e400 con 0x55eb4a61b400 2024-04-27T05:21:38.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.056+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'121 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb4a61b400 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.056+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e000 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.057+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'33 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.057+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f000 con 0x55eb49b03800 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.058+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'30 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.058+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f400 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.059+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.059+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e800 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.060+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'32 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.060+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2f800 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.061+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.061+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b2e400 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.062+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'37 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.062+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03000 con 0x55eb49b02400 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.063+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.063+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b03400 con 0x55eb49b02c00 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.063+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'26 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.063+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b02800 con 0x55eb49b02400 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.065+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'28 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.065+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5400 con 0x55eb49b02c00 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.065+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/2613031087 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'34 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49b02c00 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.065+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5ec00 con 0x55eb49b02400 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.066+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/2613031087 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'45 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49b02400 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.066+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49a5f000 con 0x55eb49b03800 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.067+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49b03800 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.067+0000 7f9def683640 20 reqs_thread_entry: start 2024-04-27T05:21:38.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.067+0000 7f9e0e719880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.067+0000 7f9e0e719880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9ded67f640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9dece5d640 20 reqs_thread_entry: start 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b80000 con 0x55eb49a3bc00 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.068+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b80400 con 0x55eb49a3bc00 2024-04-27T05:21:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.069+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(63 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.069+0000 7f9df3e8c640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.069+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94469040475392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b80800 con 0x55eb4a4b5000 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.070+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94469040475392] v29'122 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.070+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94469040466176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b80c00 con 0x55eb49a3b000 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.071+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94469040466176] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.071+0000 7f9e0e719880 20 remove_watcher() i=0 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.071+0000 7f9e0e719880 2 removed watcher, disabling cache 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.071+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94469040467328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b81000 con 0x55eb4a4b5000 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.072+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94469040467328] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.072+0000 7f9e0e719880 20 remove_watcher() i=1 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.072+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94469040468480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b81400 con 0x55eb49a3b800 2024-04-27T05:21:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.073+0000 7f9e0cebe640 1 -- 172.21.15.138:0/443774368 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94469040468480] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54d80 con 0x55eb49a3b800 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.073+0000 7f9e0e719880 20 remove_watcher() i=2 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.073+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94469040469632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b81800 con 0x55eb49a3b000 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.074+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94469040469632] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a54fc0 con 0x55eb49a3b000 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.074+0000 7f9e0e719880 20 remove_watcher() i=3 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.074+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94469040470784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b81c00 con 0x55eb4a4b5000 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.075+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94469040470784] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb4a4b5000 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.075+0000 7f9e0e719880 20 remove_watcher() i=4 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.075+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94469040471936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b81400 con 0x55eb49a3bc00 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.076+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94469040471936] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.076+0000 7f9e0e719880 20 remove_watcher() i=5 2024-04-27T05:21:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.076+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94469040473088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb49b80000 con 0x55eb49a3bc00 2024-04-27T05:21:38.095 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.084+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94469040473088] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.084+0000 7f9e0e719880 20 remove_watcher() i=6 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.084+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94469040474240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55eb4a4b5400 con 0x55eb49a3bc00 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0d6bf640 1 -- 172.21.15.138:0/443774368 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94469040474240] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55eb49a546c0 con 0x55eb49a3bc00 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 20 remove_watcher() i=7 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49a3bc00 msgr2=0x55eb499a1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49a3bc00 0x55eb499a1180 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a4b5000 msgr2=0x55eb4999fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a4b5000 0x55eb4999fb80 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49a3b000 msgr2=0x55eb499a0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49a3b000 0x55eb499a0c00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49a3b800 msgr2=0x55eb499a0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49a3b800 0x55eb499a0100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb49a3a000 msgr2=0x55eb4999f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb49a3a000 0x55eb4999f600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55eb4a628a80 tx=0x55eb499fe120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 msgr2=0x55eb4999eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 --2- 172.21.15.138:0/443774368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb4a4b4c00 0x55eb4999eb00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x55eb4a4c15c0 tx=0x55eb497cf9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 reap_dead start 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 shutdown_connections 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0c6bd640 1 -- 172.21.15.138:0/443774368 reap_dead start 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 >> 172.21.15.138:0/443774368 conn(0x55eb496bd400 msgr2=0x55eb497c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 shutdown_connections 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.085+0000 7f9e0e719880 1 -- 172.21.15.138:0/443774368 wait complete. 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49b02400 msgr2=0x55eb49afe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55eb49b02400 0x55eb49afe580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a61b400 msgr2=0x55eb49afe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55eb4a61b400 0x55eb49afe000 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49b03800 msgr2=0x55eb49aff080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55eb49b03800 0x55eb49aff080 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49b02c00 msgr2=0x55eb49afeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55eb49b02c00 0x55eb49afeb00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a61b800 msgr2=0x55eb499a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a61b800 0x55eb499a1700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55eb49a1d230 tx=0x55eb499fe8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 msgr2=0x55eb499a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0cebe640 1 -- 172.21.15.138:0/2613031087 reap_dead start 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55eb49a1d860 tx=0x55eb499feae0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 shutdown_connections 2024-04-27T05:21:38.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/2613031087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb49a3ac00 0x55eb499a0680 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 >> 172.21.15.138:0/2613031087 conn(0x55eb49a5f800 msgr2=0x55eb497c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 shutdown_connections 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/2613031087 wait complete. 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a4b5c00 msgr2=0x55eb4999f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a4b5c00 0x55eb4999f080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55eb4a5bc2a0 tx=0x55eb497cf920 comp rx=0 tx=0).stop 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 msgr2=0x55eb4999e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55eb4a578de0 tx=0x55eb497cec60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 shutdown_connections 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55eb4a4b5c00 0x55eb4999f080 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 --2- 172.21.15.138:0/967022096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55eb496bd000 0x55eb4999e580 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 >> 172.21.15.138:0/967022096 conn(0x55eb496bcc00 msgr2=0x55eb497c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 shutdown_connections 2024-04-27T05:21:38.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.086+0000 7f9e0e719880 1 -- 172.21.15.138:0/967022096 wait complete. 2024-04-27T05:21:38.100 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-27T05:21:38.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-27T05:21:38.100 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-04-27T05:21:38.181 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.181 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.179+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.179+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.179+0000 7f4badf32880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c2107a3400 0x55c210a84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.179+0000 7f4badf32880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c210a7a600 con 0x55c2107a3000 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.179+0000 7f4badf32880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c2107e2280 con 0x55c2107a3400 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.180+0000 7f4ba9ffe640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c2107a3400 0x55c210a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.180+0000 7f4baa7ff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55c2107a3000 0x55c2108aa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42808/0 (socket says 172.21.15.138:42808) 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.180+0000 7f4baa7ff640 1 -- 172.21.15.138:0/2901103809 learned_addr learned my addr 172.21.15.138:0/2901103809 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.180+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/2901103809 >> v1:172.21.15.114:6789/0 conn(0x55c2107a3000 legacy=0x55c2108aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.180+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/2901103809 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2107a7380 con 0x55c2107a3400 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/2901103809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3400 0x55c210a84580 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55c2108aec00 tx=0x55c2108b4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf32fc215bc4d77f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4ba97fd640 1 -- 172.21.15.138:0/2901103809 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bbce0 con 0x55c2107a3400 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4ba97fd640 1 -- 172.21.15.138:0/2901103809 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c210a7bb00 con 0x55c2107a3400 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4ba97fd640 1 -- 172.21.15.138:0/2901103809 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bbe40 con 0x55c2107a3400 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 -- 172.21.15.138:0/2901103809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3400 msgr2=0x55c210a84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 --2- 172.21.15.138:0/2901103809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3400 0x55c210a84580 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55c2108aec00 tx=0x55c2108b4de0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 -- 172.21.15.138:0/2901103809 shutdown_connections 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 --2- 172.21.15.138:0/2901103809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3400 0x55c210a84580 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 -- 172.21.15.138:0/2901103809 >> 172.21.15.138:0/2901103809 conn(0x55c2107a2c00 msgr2=0x55c2108aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 -- 172.21.15.138:0/2901103809 shutdown_connections 2024-04-27T05:21:38.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.182+0000 7f4badf32880 1 -- 172.21.15.138:0/2901103809 wait complete. 2024-04-27T05:21:38.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4badf32880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4badf32880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107e2780 con 0x55c2107a3000 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4baa7ff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4baa7ff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59104/0 (socket says 172.21.15.138:59104) 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4baa7ff640 1 -- 172.21.15.138:0/2166863825 learned_addr learned my addr 172.21.15.138:0/2166863825 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.183+0000 7f4baa7ff640 1 -- 172.21.15.138:0/2166863825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2107a6ea0 con 0x55c2107a3000 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/2166863825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55c211652cc0 tx=0x55c2108b4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6329a731fa9a986a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4ba8ffc640 1 -- 172.21.15.138:0/2166863825 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bb4a0 con 0x55c2107a3000 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4ba8ffc640 1 -- 172.21.15.138:0/2166863825 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c210a7be00 con 0x55c2107a3000 2024-04-27T05:21:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4ba8ffc640 1 -- 172.21.15.138:0/2166863825 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bbce0 con 0x55c2107a3000 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 -- 172.21.15.138:0/2166863825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 msgr2=0x55c210a84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 --2- 172.21.15.138:0/2166863825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55c211652cc0 tx=0x55c2108b4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 -- 172.21.15.138:0/2166863825 shutdown_connections 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 --2- 172.21.15.138:0/2166863825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 -- 172.21.15.138:0/2166863825 >> 172.21.15.138:0/2166863825 conn(0x55c2107a2c00 msgr2=0x55c2108ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 -- 172.21.15.138:0/2166863825 shutdown_connections 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 -- 172.21.15.138:0/2166863825 wait complete. 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.184+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4badf32880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4badf32880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107e28c0 con 0x55c2107a3000 2024-04-27T05:21:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4baa7ff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4baa7ff640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59120/0 (socket says 172.21.15.138:59120) 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4baa7ff640 1 -- 172.21.15.138:0/3081370122 learned_addr learned my addr 172.21.15.138:0/3081370122 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4baa7ff640 1 -- 172.21.15.138:0/3081370122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2107a76c0 con 0x55c2107a3000 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55c211652f00 tx=0x55c2108b4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4ba77f9640 1 -- 172.21.15.138:0/3081370122 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bbe40 con 0x55c2107a3000 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4ba77f9640 1 -- 172.21.15.138:0/3081370122 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c211660000 con 0x55c2107a3000 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4ba77f9640 1 -- 172.21.15.138:0/3081370122 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bb4a0 con 0x55c2107a3000 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2107a71e0 con 0x55c2107a3000 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.185+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c210814ea0 con 0x55c2107a3000 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4badf32880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4badf32880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107c6280 con 0x55c211597800 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4ba77f9640 1 -- 172.21.15.138:0/3081370122 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c2107cd600 con 0x55c2107a3000 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4ba77f9640 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2116f4000 0x55c210a85080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4ba77f9640 1 -- 172.21.15.138:0/3081370122 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c2107b5a40 con 0x55c2107a3000 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4bab000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4bab000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59126/0 (socket says 172.21.15.138:59126) 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4bab000640 1 -- 172.21.15.138:0/1326430130 learned_addr learned my addr 172.21.15.138:0/1326430130 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.186+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2116f4000 0x55c210a85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4bab000640 1 -- 172.21.15.138:0/1326430130 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c210814d00 con 0x55c211597800 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2116f4000 0x55c210a85080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55c2116a6480 tx=0x55c2108b54a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4bab000640 1 --2- 172.21.15.138:0/1326430130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55c211653f50 tx=0x55c2108b5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbab1d974fc873a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4ba4ff4640 1 -- 172.21.15.138:0/1326430130 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bbe40 con 0x55c211597800 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4ba4ff4640 1 -- 172.21.15.138:0/1326430130 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c211660300 con 0x55c211597800 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4ba4ff4640 1 -- 172.21.15.138:0/1326430130 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bb4a0 con 0x55c211597800 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 -- 172.21.15.138:0/1326430130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 msgr2=0x55c210a84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 --2- 172.21.15.138:0/1326430130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55c211653f50 tx=0x55c2108b5c20 comp rx=0 tx=0).stop 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 -- 172.21.15.138:0/1326430130 shutdown_connections 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 --2- 172.21.15.138:0/1326430130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 -- 172.21.15.138:0/1326430130 >> 172.21.15.138:0/1326430130 conn(0x55c2107a3400 msgr2=0x55c2108ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 -- 172.21.15.138:0/1326430130 shutdown_connections 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.187+0000 7f4badf32880 1 -- 172.21.15.138:0/1326430130 wait complete. 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4badf32880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4badf32880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107c7180 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4bab000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4bab000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59128/0 (socket says 172.21.15.138:59128) 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 learned_addr learned my addr 172.21.15.138:0/411880040 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c210814b60 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.188+0000 7f4bab000640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c211653950 tx=0x55c2108b51a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4ba37f1640 1 -- 172.21.15.138:0/411880040 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bbe40 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4ba37f1640 1 -- 172.21.15.138:0/411880040 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c211660480 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2108149c0 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4ba37f1640 1 -- 172.21.15.138:0/411880040 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2107bb4a0 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c210814820 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4ba37f1640 1 -- 172.21.15.138:0/411880040 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c2107cdb80 con 0x55c211597800 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4ba37f1640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c210b28000 0x55c210a85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4ba37f1640 1 -- 172.21.15.138:0/411880040 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c2107b5c00 con 0x55c211597800 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.189+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c210b28000 0x55c210a85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c211597c00 0x55c210a85b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28400 con 0x55c211597c00 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4bab000640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c211597c00 0x55c210a85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c210b28000 0x55c210a85600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55c2108aec00 tx=0x55c210aee120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4bab000640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c211597c00 0x55c210a85b80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 20 realm 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28800 con 0x55c211597c00 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.190+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28c00 con 0x55c211597c00 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29000 con 0x55c211597c00 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c210b29400 0x55c210a86100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28400 con 0x55c210b29400 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c210b29400 0x55c210a86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c210b29400 0x55c210a86100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 20 searching for the correct realm 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.191+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28800 con 0x55c210b29400 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.192+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.192+0000 7f4baa7ff640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c210b28800 0x55c210a86c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.192+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28400 con 0x55c210b28800 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.192+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c210b28800 0x55c210a86c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.192+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c210b28800 0x55c210a86c00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c210b28400 0x55c210a87180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4bab000640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c210b28400 0x55c210a87180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4bab000640 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c210b28400 0x55c210a87180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.193+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28800 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.194+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28800 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.195+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.196+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28800 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.197+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28800 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.198+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b29400 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28400 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c211597c00 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211596800 con 0x55c210b28800 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28c00 con 0x55c211597c00 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.199+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29000 con 0x55c211597c00 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29800 con 0x55c210b29400 2024-04-27T05:21:38.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 zone default found 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 4 Realm: () 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 zonegroup default 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29c00 con 0x55c210b28400 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.200+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586000 con 0x55c210b29400 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586400 con 0x55c210b29400 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 20 started sync module instance, tier type = 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.201+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586800 con 0x55c210b28800 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.202+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.202+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94291991447808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586c00 con 0x55c210b28800 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.202+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211587000 con 0x55c211597c00 2024-04-27T05:21:38.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.203+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94291991447808] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.203+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.203+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94291991448960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211587400 con 0x55c211597c00 2024-04-27T05:21:38.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.203+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211587800 con 0x55c210b29400 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.204+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.204+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94291991450112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586000 con 0x55c210b29400 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.204+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586400 con 0x55c210b28800 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.204+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94291991448960] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.205+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.205+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94291991451264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211587800 con 0x55c210b28800 2024-04-27T05:21:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.205+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29800 con 0x55c211597c00 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.205+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94291991450112] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.206+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94291991451264] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.206+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.206+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94291991452416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b28c00 con 0x55c211597c00 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.206+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29000 con 0x55c210b28400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.207+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94291991452416] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.207+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.207+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94291991453568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29c00 con 0x55c210b28400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.207+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c2116f4c00 con 0x55c210b28400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.208+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.208+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94291991454720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586800 con 0x55c210b28400 2024-04-27T05:21:38.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.208+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586c00 con 0x55c210b28400 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.208+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94291991453568] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94291991454720] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94291991455872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211586400 con 0x55c210b28400 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=0 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=1 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=2 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=3 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=4 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=5 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.209+0000 7f4badf32880 20 add_watcher() i=6 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.210+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94291991455872] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.210+0000 7f4badf32880 20 add_watcher() i=7 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.210+0000 7f4badf32880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.210+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211587800 con 0x55c211597c00 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.211+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.211+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94291991457024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c211587000 con 0x55c211597c00 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94291991457024] v29'123 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4b917cd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4badf32880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4badf32880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4badf32880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4badf32880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107c6780 con 0x55c211587c00 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4ba9ffe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4ba9ffe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59138/0 (socket says 172.21.15.138:59138) 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.213+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/1709979478 learned_addr learned my addr 172.21.15.138:0/1709979478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.214+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/1709979478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c210814680 con 0x55c211587c00 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.214+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/1709979478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55c211714780 tx=0x55c210aee0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74b509e860a391d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.214+0000 7f4b90fcc640 1 -- 172.21.15.138:0/1709979478 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c211588580 con 0x55c211587c00 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.214+0000 7f4b90fcc640 1 -- 172.21.15.138:0/1709979478 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c211660000 con 0x55c211587c00 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.214+0000 7f4b90fcc640 1 -- 172.21.15.138:0/1709979478 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2115886e0 con 0x55c211587c00 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- 172.21.15.138:0/1709979478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 msgr2=0x55c210a86680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 --2- 172.21.15.138:0/1709979478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55c211714780 tx=0x55c210aee0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- 172.21.15.138:0/1709979478 shutdown_connections 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 --2- 172.21.15.138:0/1709979478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- 172.21.15.138:0/1709979478 >> 172.21.15.138:0/1709979478 conn(0x55c211587400 msgr2=0x55c2108ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- 172.21.15.138:0/1709979478 shutdown_connections 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- 172.21.15.138:0/1709979478 wait complete. 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 Processor -- start 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- start start 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4badf32880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107c6500 con 0x55c211587c00 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4ba9ffe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4ba9ffe640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59142/0 (socket says 172.21.15.138:59142) 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.215+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/2382474602 learned_addr learned my addr 172.21.15.138:0/2382474602 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/2382474602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2108144e0 con 0x55c211587c00 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4ba9ffe640 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55c210b09740 tx=0x55c210aee000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4b8f7c9640 1 -- 172.21.15.138:0/2382474602 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c211588840 con 0x55c211587c00 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2107a6ea0 con 0x55c211587c00 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4b8f7c9640 1 -- 172.21.15.138:0/2382474602 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c211660780 con 0x55c211587c00 2024-04-27T05:21:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4b8f7c9640 1 -- 172.21.15.138:0/2382474602 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c211588580 con 0x55c211587c00 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.216+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2107a76c0 con 0x55c211587c00 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4b8f7c9640 1 -- 172.21.15.138:0/2382474602 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c2107ccb00 con 0x55c211587c00 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4b8f7c9640 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c211586800 0x55c210a87700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4b8f7c9640 1 -- 172.21.15.138:0/2382474602 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55c2107b5dc0 con 0x55c211587c00 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4badf32880 5 note: GC not initialized 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4bab000640 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c211586800 0x55c210a87700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4b8cfc4640 20 reqs_thread_entry: start 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4badf32880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4bab000640 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c211586800 0x55c210a87700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c211714ba0 tx=0x55c210aee960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4badf32880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4b8afc0640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4b8a79e640 20 reqs_thread_entry: start 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4badf32880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.217+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16000 con 0x55c210b28400 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16400 con 0x55c210b28800 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:38.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.218+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16800 con 0x55c210b28800 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.219+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16c00 con 0x55c210b28400 2024-04-27T05:21:38.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(65 fud.buckets [call] v29'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17000 con 0x55c210b28800 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.220+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17800 con 0x55c210b28800 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v29'21 uv21 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4badf32880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4badf32880 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4badf32880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94291991459328 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17c00 con 0x55c210b28400 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== watch-notify(notify (1) cookie 94291991454720 notify 124554051597 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x55c210aedd40 con 0x55c210b28400 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4ba4ff4640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051597 cookie 94291991454720 notifier 4557 bl.length()=338 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4ba4ff4640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4ba4ff4640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(68 notify.6 [notify cookie 94291991459328 out=8b] v0'0 uv120 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4ba4ff4640 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210b29c00 con 0x55c210b28400 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.222+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== watch-notify(notify_complete (2) cookie 94291991459328 notify 124554051597 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c210b9f860 con 0x55c210b28400 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.223+0000 7f4b917cd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.223+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94291991457024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16400 con 0x55c211597c00 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.224+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94291991457024] v29'124 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.224+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94291991447808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16800 con 0x55c210b28800 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.225+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94291991447808] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.225+0000 7f4badf32880 20 remove_watcher() i=0 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.225+0000 7f4badf32880 2 removed watcher, disabling cache 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.225+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94291991448960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17000 con 0x55c211597c00 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.226+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94291991448960] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.226+0000 7f4badf32880 20 remove_watcher() i=1 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.226+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94291991450112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17800 con 0x55c210b29400 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.227+0000 7f4baa7ff640 1 -- 172.21.15.138:0/411880040 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94291991450112] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42fc0 con 0x55c210b29400 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.227+0000 7f4badf32880 20 remove_watcher() i=2 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.227+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94291991451264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17400 con 0x55c210b28800 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.228+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94291991451264] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b43200 con 0x55c210b28800 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.228+0000 7f4badf32880 20 remove_watcher() i=3 2024-04-27T05:21:38.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.228+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94291991452416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16000 con 0x55c211597c00 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.229+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94291991452416] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c211597c00 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.229+0000 7f4badf32880 20 remove_watcher() i=4 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.229+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94291991453568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16c00 con 0x55c210b28400 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.230+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94291991453568] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.230+0000 7f4badf32880 20 remove_watcher() i=5 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.230+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94291991454720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c17c00 con 0x55c210b28400 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.231+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94291991454720] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.231+0000 7f4badf32880 20 remove_watcher() i=6 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.231+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94291991455872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c210c16400 con 0x55c210b28400 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4bab000640 1 -- 172.21.15.138:0/411880040 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94291991455872] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c210b42900 con 0x55c210b28400 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 20 remove_watcher() i=7 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c210b28400 msgr2=0x55c210a87180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c210b28400 0x55c210a87180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c211597c00 msgr2=0x55c210a85b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c211597c00 0x55c210a85b80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c210b28800 msgr2=0x55c210a86c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c210b28800 0x55c210a86c00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c210b29400 msgr2=0x55c210a86100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c210b29400 0x55c210a86100 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c210b28000 msgr2=0x55c210a85600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c210b28000 0x55c210a85600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55c2108aec00 tx=0x55c210aee120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 msgr2=0x55c210a84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/411880040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211597800 0x55c210a84b00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c211653950 tx=0x55c2108b51a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 reap_dead start 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4ba9ffe640 1 -- 172.21.15.138:0/411880040 reap_dead start 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 shutdown_connections 2024-04-27T05:21:38.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 >> 172.21.15.138:0/411880040 conn(0x55c2107a3400 msgr2=0x55c2108ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 shutdown_connections 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/411880040 wait complete. 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c211586800 msgr2=0x55c210a87700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c211586800 0x55c210a87700 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c211714ba0 tx=0x55c210aee960 comp rx=0 tx=0).stop 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 msgr2=0x55c210a86680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55c210b09740 tx=0x55c210aee000 comp rx=0 tx=0).stop 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 shutdown_connections 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c211586800 0x55c210a87700 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 --2- 172.21.15.138:0/2382474602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c211587c00 0x55c210a86680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 >> 172.21.15.138:0/2382474602 conn(0x55c211587400 msgr2=0x55c2108ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 shutdown_connections 2024-04-27T05:21:38.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.232+0000 7f4badf32880 1 -- 172.21.15.138:0/2382474602 wait complete. 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2116f4000 msgr2=0x55c210a85080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2116f4000 0x55c210a85080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55c2116a6480 tx=0x55c2108b54a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 msgr2=0x55c210a84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55c211652f00 tx=0x55c2108b4c60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 shutdown_connections 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c2116f4000 0x55c210a85080 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 --2- 172.21.15.138:0/3081370122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2107a3000 0x55c210a84580 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 >> 172.21.15.138:0/3081370122 conn(0x55c2107a2c00 msgr2=0x55c2108ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 shutdown_connections 2024-04-27T05:21:38.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.233+0000 7f4badf32880 1 -- 172.21.15.138:0/3081370122 wait complete. 2024-04-27T05:21:38.256 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1'] 2024-04-27T05:21:38.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1'] 2024-04-27T05:21:38.256 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f330240f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f330240f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ae53c70600 con 0x55ae53999400 2024-04-27T05:21:38.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f330240f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ae539d8280 con 0x55ae53999000 2024-04-27T05:21:38.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f32fe8dc640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.300+0000 7f32fe0db640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55ae53999400 0x55ae53aa0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42816/0 (socket says 172.21.15.138:42816) 2024-04-27T05:21:38.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.301+0000 7f32fe0db640 1 -- 172.21.15.138:0/927827538 learned_addr learned my addr 172.21.15.138:0/927827538 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.301+0000 7f32fe8dc640 1 -- 172.21.15.138:0/927827538 >> v1:172.21.15.114:6789/0 conn(0x55ae53999400 legacy=0x55ae53aa0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.301+0000 7f32fe8dc640 1 -- 172.21.15.138:0/927827538 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5399d380 con 0x55ae53999000 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.302+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/927827538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55ae53aa4d50 tx=0x55ae53aaade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cfb6bbe290732d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.302+0000 7f32fd8da640 1 -- 172.21.15.138:0/927827538 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b1ce0 con 0x55ae53999000 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f32fd8da640 1 -- 172.21.15.138:0/927827538 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae53c71b00 con 0x55ae53999000 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f32fd8da640 1 -- 172.21.15.138:0/927827538 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b1e40 con 0x55ae53999000 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 -- 172.21.15.138:0/927827538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 msgr2=0x55ae53c7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 --2- 172.21.15.138:0/927827538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55ae53aa4d50 tx=0x55ae53aaade0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 -- 172.21.15.138:0/927827538 shutdown_connections 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 --2- 172.21.15.138:0/927827538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 -- 172.21.15.138:0/927827538 >> 172.21.15.138:0/927827538 conn(0x55ae53998c00 msgr2=0x55ae53aa0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 -- 172.21.15.138:0/927827538 shutdown_connections 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.303+0000 7f330240f880 1 -- 172.21.15.138:0/927827538 wait complete. 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f330240f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f330240f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae539d8780 con 0x55ae53999000 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fe8dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fe8dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59160/0 (socket says 172.21.15.138:59160) 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fe8dc640 1 -- 172.21.15.138:0/554657463 learned_addr learned my addr 172.21.15.138:0/554657463 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fe8dc640 1 -- 172.21.15.138:0/554657463 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5399cea0 con 0x55ae53999000 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/554657463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55ae5479d560 tx=0x55ae53aaacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fd0ba8301db142 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fd0d9640 1 -- 172.21.15.138:0/554657463 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b14a0 con 0x55ae53999000 2024-04-27T05:21:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fd0d9640 1 -- 172.21.15.138:0/554657463 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae53c71e00 con 0x55ae53999000 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.304+0000 7f32fd0d9640 1 -- 172.21.15.138:0/554657463 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b1ce0 con 0x55ae53999000 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- 172.21.15.138:0/554657463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 msgr2=0x55ae53c7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 --2- 172.21.15.138:0/554657463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55ae5479d560 tx=0x55ae53aaacc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- 172.21.15.138:0/554657463 shutdown_connections 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 --2- 172.21.15.138:0/554657463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- 172.21.15.138:0/554657463 >> 172.21.15.138:0/554657463 conn(0x55ae53998c00 msgr2=0x55ae53aa1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- 172.21.15.138:0/554657463 shutdown_connections 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- 172.21.15.138:0/554657463 wait complete. 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f330240f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae539d88c0 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.305+0000 7f32fe8dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fe8dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59170/0 (socket says 172.21.15.138:59170) 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fe8dc640 1 -- 172.21.15.138:0/986812151 learned_addr learned my addr 172.21.15.138:0/986812151 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fe8dc640 1 -- 172.21.15.138:0/986812151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5399d6c0 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55ae54854de0 tx=0x55ae53aaac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fb8d6640 1 -- 172.21.15.138:0/986812151 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b1e40 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fb8d6640 1 -- 172.21.15.138:0/986812151 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae54778000 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae5399d1e0 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f32fb8d6640 1 -- 172.21.15.138:0/986812151 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b14a0 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.306+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae53a0aea0 con 0x55ae53999000 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f330240f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f330240f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae539bc280 con 0x55ae54790c00 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32fb8d6640 1 -- 172.21.15.138:0/986812151 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ae539c2b00 con 0x55ae53999000 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32fb8d6640 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae54791c00 0x55ae53c7b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32fb8d6640 1 -- 172.21.15.138:0/986812151 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55ae539aba40 con 0x55ae53999000 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32ff0dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32ff0dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59184/0 (socket says 172.21.15.138:59184) 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32ff0dd640 1 -- 172.21.15.138:0/2307018199 learned_addr learned my addr 172.21.15.138:0/2307018199 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae54791c00 0x55ae53c7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32ff0dd640 1 -- 172.21.15.138:0/2307018199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae53a0ad00 con 0x55ae54790c00 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/2307018199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55ae54855d10 tx=0x55ae53aabc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11be8985a19b662 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae54791c00 0x55ae53c7b080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55ae548982a0 tx=0x55ae53aaacc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.307+0000 7f32f90d1640 1 -- 172.21.15.138:0/2307018199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b1e40 con 0x55ae54790c00 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f32f90d1640 1 -- 172.21.15.138:0/2307018199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae54778600 con 0x55ae54790c00 2024-04-27T05:21:38.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f32f90d1640 1 -- 172.21.15.138:0/2307018199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b14a0 con 0x55ae54790c00 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- 172.21.15.138:0/2307018199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 msgr2=0x55ae53c7ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 --2- 172.21.15.138:0/2307018199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55ae54855d10 tx=0x55ae53aabc20 comp rx=0 tx=0).stop 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- 172.21.15.138:0/2307018199 shutdown_connections 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 --2- 172.21.15.138:0/2307018199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- 172.21.15.138:0/2307018199 >> 172.21.15.138:0/2307018199 conn(0x55ae53999400 msgr2=0x55ae53aa3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- 172.21.15.138:0/2307018199 shutdown_connections 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- 172.21.15.138:0/2307018199 wait complete. 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f330240f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae539bd180 con 0x55ae54790c00 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f32ff0dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f32ff0dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59198/0 (socket says 172.21.15.138:59198) 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.308+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 learned_addr learned my addr 172.21.15.138:0/4266461114 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae53a0ab60 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55ae5479d5c0 tx=0x55ae53aab1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f32f78ce640 1 -- 172.21.15.138:0/4266461114 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b1e40 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f32f78ce640 1 -- 172.21.15.138:0/4266461114 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae54778780 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae53a0a9c0 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f32f78ce640 1 -- 172.21.15.138:0/4266461114 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae539b14a0 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.309+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae53a0a820 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32f78ce640 1 -- 172.21.15.138:0/4266461114 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ae539c3600 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32f78ce640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae53d18000 0x55ae53c7b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32f78ce640 1 -- 172.21.15.138:0/4266461114 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55ae539abc00 con 0x55ae54790c00 2024-04-27T05:21:38.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32fe0db640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae53d18000 0x55ae53c7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ae54791000 0x55ae53c7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae54791400 con 0x55ae54791000 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ae54791000 0x55ae53c7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32fe0db640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae53d18000 0x55ae53c7b600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55ae54906f60 tx=0x55ae53cde120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.310+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ae54791000 0x55ae53c7bb80 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 realm 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae54791800 con 0x55ae54791000 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19000 con 0x55ae54791000 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19400 con 0x55ae54791000 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ae53d19800 0x55ae53c7c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19c00 con 0x55ae53d19800 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.311+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ae53d19800 0x55ae53c7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ae53d19800 0x55ae53c7c100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f330240f880 20 searching for the correct realm 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19000 con 0x55ae53d19800 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f32fe8dc640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ae53d19000 0x55ae53c7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19c00 con 0x55ae53d19000 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.312+0000 7f32fe0db640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ae53d19000 0x55ae53c7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.313+0000 7f32fe0db640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ae53d19000 0x55ae53c7cc00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.313+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.313+0000 7f32fe0db640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ae53d19c00 0x55ae53c7d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.313+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.313+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ae53d19c00 0x55ae53c7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.313+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ae53d19c00 0x55ae53c7d180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.314+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19000 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.315+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19000 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.316+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.316+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.316+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.316+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.316+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.316+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.317+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19000 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.318+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19000 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.319+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19800 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19c00 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae54791000 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d18c00 con 0x55ae53d19000 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f330240f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f330240f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f330240f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f330240f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19400 con 0x55ae54791000 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.320+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae54791400 con 0x55ae54791000 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae54791800 con 0x55ae53d19800 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 zone default found 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 4 Realm: () 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 zonegroup default 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40000 con 0x55ae53d19c00 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40400 con 0x55ae53d19800 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.321+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40800 con 0x55ae53d19800 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f330240f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f330240f880 20 started sync module instance, tier type = 2024-04-27T05:21:38.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f330240f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.322+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40c00 con 0x55ae53d19000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.323+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.323+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94207218231552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41000 con 0x55ae53d19000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.323+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41400 con 0x55ae54791000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.324+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94207218231552] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.324+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.324+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94207218232704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41800 con 0x55ae54791000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.324+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41c00 con 0x55ae53d19800 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.325+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.325+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94207218233856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40400 con 0x55ae53d19800 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.325+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40800 con 0x55ae53d19000 2024-04-27T05:21:38.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.325+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94207218232704] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.326+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94207218233856] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.326+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.326+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94207218235008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41c00 con 0x55ae53d19000 2024-04-27T05:21:38.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.326+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40000 con 0x55ae54791000 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.327+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94207218235008] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.327+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.327+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94207218236160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae54791800 con 0x55ae54791000 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.327+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae54791400 con 0x55ae53d19c00 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.328+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94207218236160] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.328+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.328+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94207218237312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d19400 con 0x55ae53d19c00 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.328+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae548f9800 con 0x55ae53d19c00 2024-04-27T05:21:38.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.329+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94207218237312] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.329+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.329+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94207218238464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae548f9400 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.329+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40c00 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94207218239616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41000 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=0 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=1 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=2 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=3 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=4 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=5 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94207218238464] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.330+0000 7f330240f880 20 add_watcher() i=6 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.332+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94207218239616] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.332+0000 7f330240f880 20 add_watcher() i=7 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.332+0000 7f330240f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.332+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d40800 con 0x55ae54791000 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.332+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.332+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94207218240768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d41400 con 0x55ae54791000 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.333+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94207218240768] v29'125 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.333+0000 7f32e58aa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.333+0000 7f330240f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.333+0000 7f330240f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.333+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.333+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f330240f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f330240f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae539bc780 con 0x55ae53d18c00 2024-04-27T05:21:38.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32fe0db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32fe0db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59204/0 (socket says 172.21.15.138:59204) 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32fe0db640 1 -- 172.21.15.138:0/1356582193 learned_addr learned my addr 172.21.15.138:0/1356582193 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32fe0db640 1 -- 172.21.15.138:0/1356582193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae53a0a680 con 0x55ae53d18c00 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32fe0db640 1 --2- 172.21.15.138:0/1356582193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55ae53cf89c0 tx=0x55ae53cde180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342af8ba11b3cc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32e50a9640 1 -- 172.21.15.138:0/1356582193 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae53d42580 con 0x55ae53d18c00 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32e50a9640 1 -- 172.21.15.138:0/1356582193 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae54778000 con 0x55ae53d18c00 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.334+0000 7f32e50a9640 1 -- 172.21.15.138:0/1356582193 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae53d426e0 con 0x55ae53d18c00 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 -- 172.21.15.138:0/1356582193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 msgr2=0x55ae53c7c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 --2- 172.21.15.138:0/1356582193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55ae53cf89c0 tx=0x55ae53cde180 comp rx=0 tx=0).stop 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 -- 172.21.15.138:0/1356582193 shutdown_connections 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 --2- 172.21.15.138:0/1356582193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 -- 172.21.15.138:0/1356582193 >> 172.21.15.138:0/1356582193 conn(0x55ae53d41800 msgr2=0x55ae53aa3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 -- 172.21.15.138:0/1356582193 shutdown_connections 2024-04-27T05:21:38.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 -- 172.21.15.138:0/1356582193 wait complete. 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 Processor -- start 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.335+0000 7f330240f880 1 -- start start 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f330240f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f330240f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae539bc500 con 0x55ae53d18c00 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32fe0db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32fe0db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59214/0 (socket says 172.21.15.138:59214) 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32fe0db640 1 -- 172.21.15.138:0/3406008562 learned_addr learned my addr 172.21.15.138:0/3406008562 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32fe0db640 1 -- 172.21.15.138:0/3406008562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae53a0a4e0 con 0x55ae53d18c00 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32fe0db640 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55ae53cf9890 tx=0x55ae53cdeae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32e38a6640 1 -- 172.21.15.138:0/3406008562 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae53d42840 con 0x55ae53d18c00 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32e38a6640 1 -- 172.21.15.138:0/3406008562 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae54778a80 con 0x55ae53d18c00 2024-04-27T05:21:38.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae5399d380 con 0x55ae53d18c00 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f32e38a6640 1 -- 172.21.15.138:0/3406008562 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ae53d42580 con 0x55ae53d18c00 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.336+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae5399cea0 con 0x55ae53d18c00 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.337+0000 7f32e38a6640 1 -- 172.21.15.138:0/3406008562 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ae539c3b80 con 0x55ae53d18c00 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.337+0000 7f32e38a6640 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae548f9800 0x55ae53c7d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.337+0000 7f32e38a6640 1 -- 172.21.15.138:0/3406008562 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55ae539abdc0 con 0x55ae53d18c00 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.337+0000 7f330240f880 5 note: GC not initialized 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.337+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae548f9800 0x55ae53c7d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.337+0000 7f32e10a1640 20 reqs_thread_entry: start 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f32ff0dd640 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae548f9800 0x55ae53c7d700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ae54906660 tx=0x55ae53cde8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f32df09d640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f32de87b640 20 reqs_thread_entry: start 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e10000 con 0x55ae54791000 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.338+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e10400 con 0x55ae53d19000 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e10800 con 0x55ae53d19000 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.339+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e10c00 con 0x55ae53d19c00 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.341+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(65 fud.buckets [call] v29'3 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.341+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e11400 con 0x55ae53d19000 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(66 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'5 uv5 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 10 moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94207218243072 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e11c00 con 0x55ae53d19800 2024-04-27T05:21:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94207218233856 notify 124554051589 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55ae5489ed00 con 0x55ae53d19800 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32f90d1640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051589 cookie 94207218233856 notifier 4565 bl.length()=725 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32f90d1640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32f90d1640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32f90d1640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(67 notify.2 [notify cookie 94207218243072 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32f90d1640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4a000 con 0x55ae53d19800 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94207218243072 notify 124554051589 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ae5489eea0 con 0x55ae53d19800 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_sources= new_sources= 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential sources added=[] removed=[] 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_dests= new_dests= 2024-04-27T05:21:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential dests added=[] removed=[] 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.342+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4a800 con 0x55ae53d19000 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.343+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(69 foo.buckets [call] v29'22 uv22 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.343+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4ac00 con 0x55ae53d19000 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.344+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v29'23 uv23 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.344+0000 7f330240f880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.344+0000 7f330240f880 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.344+0000 7f330240f880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.344+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94207218243072 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4b000 con 0x55ae53d19c00 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== watch-notify(notify (1) cookie 94207218238464 notify 124554051598 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55ae53cddd40 con 0x55ae53d19c00 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32f90d1640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051598 cookie 94207218238464 notifier 4565 bl.length()=314 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32f90d1640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32f90d1640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [notify cookie 94207218243072 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32f90d1640 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4a400 con 0x55ae53d19c00 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.345+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== watch-notify(notify_complete (2) cookie 94207218243072 notify 124554051598 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ae53d99a00 con 0x55ae53d19c00 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.346+0000 7f32e58aa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.346+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94207218240768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4b400 con 0x55ae54791000 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.347+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94207218240768] v29'126 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.347+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94207218231552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4b800 con 0x55ae53d19000 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.348+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94207218231552] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.348+0000 7f330240f880 20 remove_watcher() i=0 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.348+0000 7f330240f880 2 removed watcher, disabling cache 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.348+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94207218232704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4bc00 con 0x55ae54791000 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.349+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94207218232704] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.349+0000 7f330240f880 20 remove_watcher() i=1 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.349+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94207218233856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4b000 con 0x55ae53d19800 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.350+0000 7f32fe8dc640 1 -- 172.21.15.138:0/4266461114 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94207218233856] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2efc0 con 0x55ae53d19800 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.350+0000 7f330240f880 20 remove_watcher() i=2 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.350+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94207218235008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4a400 con 0x55ae53d19000 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.351+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94207218235008] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2f200 con 0x55ae53d19000 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.351+0000 7f330240f880 20 remove_watcher() i=3 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.351+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94207218236160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53d4b400 con 0x55ae54791000 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.352+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94207218236160] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae54791000 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.352+0000 7f330240f880 20 remove_watcher() i=4 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.352+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94207218237312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e10000 con 0x55ae53d19c00 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.353+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94207218237312] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.353+0000 7f330240f880 20 remove_watcher() i=5 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.353+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94207218238464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e10c00 con 0x55ae53d19c00 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.354+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94207218238464] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.354+0000 7f330240f880 20 remove_watcher() i=6 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.354+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94207218239616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ae53e11400 con 0x55ae53d19c00 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f32ff0dd640 1 -- 172.21.15.138:0/4266461114 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94207218239616] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ae53d2e900 con 0x55ae53d19c00 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 20 remove_watcher() i=7 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ae53d19c00 msgr2=0x55ae53c7d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ae53d19c00 0x55ae53c7d180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ae54791000 msgr2=0x55ae53c7bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ae54791000 0x55ae53c7bb80 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ae53d19000 msgr2=0x55ae53c7cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ae53d19000 0x55ae53c7cc00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ae53d19800 msgr2=0x55ae53c7c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ae53d19800 0x55ae53c7c100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae53d18000 msgr2=0x55ae53c7b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae53d18000 0x55ae53c7b600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55ae54906f60 tx=0x55ae53cde120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 msgr2=0x55ae53c7ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/4266461114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae54790c00 0x55ae53c7ab00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55ae5479d5c0 tx=0x55ae53aab1a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 reap_dead start 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f32fe0db640 1 -- 172.21.15.138:0/4266461114 reap_dead start 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 shutdown_connections 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 >> 172.21.15.138:0/4266461114 conn(0x55ae53999400 msgr2=0x55ae53aa3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 shutdown_connections 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/4266461114 wait complete. 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae548f9800 msgr2=0x55ae53c7d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae548f9800 0x55ae53c7d700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ae54906660 tx=0x55ae53cde8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 msgr2=0x55ae53c7c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.355+0000 7f330240f880 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55ae53cf9890 tx=0x55ae53cdeae0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 shutdown_connections 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae548f9800 0x55ae53c7d700 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 --2- 172.21.15.138:0/3406008562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53d18c00 0x55ae53c7c680 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 >> 172.21.15.138:0/3406008562 conn(0x55ae53d41800 msgr2=0x55ae53aa3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 shutdown_connections 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/3406008562 wait complete. 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae54791c00 msgr2=0x55ae53c7b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae54791c00 0x55ae53c7b080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55ae548982a0 tx=0x55ae53aaacc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 msgr2=0x55ae53c7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55ae54854de0 tx=0x55ae53aaac60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 shutdown_connections 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ae54791c00 0x55ae53c7b080 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 --2- 172.21.15.138:0/986812151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ae53999000 0x55ae53c7a580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 >> 172.21.15.138:0/986812151 conn(0x55ae53998c00 msgr2=0x55ae53aa1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 shutdown_connections 2024-04-27T05:21:38.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.356+0000 7f330240f880 1 -- 172.21.15.138:0/986812151 wait complete. 2024-04-27T05:21:38.382 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T05:21:38.382 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T05:21:38.382 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T05:21:38.425 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.425 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd24342880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5558990df400 0x5558993c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd24342880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5558993b6600 con 0x5558990df000 2024-04-27T05:21:38.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd24342880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55589911e280 con 0x5558990df400 2024-04-27T05:21:38.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd20547640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5558990df400 0x5558993c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd20d48640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5558990df000 0x5558991e6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42832/0 (socket says 172.21.15.138:42832) 2024-04-27T05:21:38.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd20d48640 1 -- 172.21.15.138:0/1087435712 learned_addr learned my addr 172.21.15.138:0/1087435712 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd20547640 1 -- 172.21.15.138:0/1087435712 >> v1:172.21.15.114:6789/0 conn(0x5558990df000 legacy=0x5558991e6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.423+0000 7fcd20547640 1 -- 172.21.15.138:0/1087435712 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558990e3380 con 0x5558990df400 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.425+0000 7fcd20547640 1 --2- 172.21.15.138:0/1087435712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df400 0x5558993c0580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5558991eac00 tx=0x5558991f0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f3a9a39c401b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.425+0000 7fcd1fd46640 1 -- 172.21.15.138:0/1087435712 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f7ce0 con 0x5558990df400 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.425+0000 7fcd1fd46640 1 -- 172.21.15.138:0/1087435712 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558993b7b00 con 0x5558990df400 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.425+0000 7fcd1fd46640 1 -- 172.21.15.138:0/1087435712 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f7e40 con 0x5558990df400 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 -- 172.21.15.138:0/1087435712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df400 msgr2=0x5558993c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 --2- 172.21.15.138:0/1087435712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df400 0x5558993c0580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5558991eac00 tx=0x5558991f0de0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 -- 172.21.15.138:0/1087435712 shutdown_connections 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 --2- 172.21.15.138:0/1087435712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df400 0x5558993c0580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 -- 172.21.15.138:0/1087435712 >> 172.21.15.138:0/1087435712 conn(0x5558990dec00 msgr2=0x5558991e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 -- 172.21.15.138:0/1087435712 shutdown_connections 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 -- 172.21.15.138:0/1087435712 wait complete. 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.426+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd24342880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd24342880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589911e780 con 0x5558990df000 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd20d48640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd20d48640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59230/0 (socket says 172.21.15.138:59230) 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd20d48640 1 -- 172.21.15.138:0/2192440609 learned_addr learned my addr 172.21.15.138:0/2192440609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd20d48640 1 -- 172.21.15.138:0/2192440609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558990e2ea0 con 0x5558990df000 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd20d48640 1 --2- 172.21.15.138:0/2192440609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x555899f9acc0 tx=0x5558991f0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92240be8b0c0440 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd1f545640 1 -- 172.21.15.138:0/2192440609 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f74a0 con 0x5558990df000 2024-04-27T05:21:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd1f545640 1 -- 172.21.15.138:0/2192440609 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558993b7e00 con 0x5558990df000 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.427+0000 7fcd1f545640 1 -- 172.21.15.138:0/2192440609 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f7ce0 con 0x5558990df000 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- 172.21.15.138:0/2192440609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 msgr2=0x5558993c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 --2- 172.21.15.138:0/2192440609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x555899f9acc0 tx=0x5558991f0cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- 172.21.15.138:0/2192440609 shutdown_connections 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 --2- 172.21.15.138:0/2192440609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- 172.21.15.138:0/2192440609 >> 172.21.15.138:0/2192440609 conn(0x5558990dec00 msgr2=0x5558991e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- 172.21.15.138:0/2192440609 shutdown_connections 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- 172.21.15.138:0/2192440609 wait complete. 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd24342880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55589911e8c0 con 0x5558990df000 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd20d48640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd20d48640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59246/0 (socket says 172.21.15.138:59246) 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.428+0000 7fcd20d48640 1 -- 172.21.15.138:0/527656535 learned_addr learned my addr 172.21.15.138:0/527656535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd20d48640 1 -- 172.21.15.138:0/527656535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558990e36c0 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd20d48640 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x555899f9af00 tx=0x5558991f0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd1dd42640 1 -- 172.21.15.138:0/527656535 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f7e40 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd1dd42640 1 -- 172.21.15.138:0/527656535 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555899fb0000 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558990e31e0 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd1dd42640 1 -- 172.21.15.138:0/527656535 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f74a0 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555899150ea0 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.429+0000 7fcd1dd42640 1 -- 172.21.15.138:0/527656535 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555899109600 con 0x5558990df000 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd24342880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd24342880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555899102280 con 0x555899ed7800 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd1dd42640 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55589a034000 0x5558993c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd1dd42640 1 -- 172.21.15.138:0/527656535 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5558990f1a40 con 0x5558990df000 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd21549640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd21549640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59252/0 (socket says 172.21.15.138:59252) 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd21549640 1 -- 172.21.15.138:0/2966500640 learned_addr learned my addr 172.21.15.138:0/2966500640 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd20d48640 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55589a034000 0x5558993c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd21549640 1 -- 172.21.15.138:0/2966500640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555899150d00 con 0x555899ed7800 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd20d48640 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55589a034000 0x5558993c1080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x555899fe2450 tx=0x5558991f14a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd21549640 1 --2- 172.21.15.138:0/2966500640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x555899f9b440 tx=0x5558991f1c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef929b714617491 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd1b53d640 1 -- 172.21.15.138:0/2966500640 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f7e40 con 0x555899ed7800 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd1b53d640 1 -- 172.21.15.138:0/2966500640 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555899fb0300 con 0x555899ed7800 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.430+0000 7fcd1b53d640 1 -- 172.21.15.138:0/2966500640 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f74a0 con 0x555899ed7800 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- 172.21.15.138:0/2966500640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 msgr2=0x5558993c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 --2- 172.21.15.138:0/2966500640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x555899f9b440 tx=0x5558991f1c20 comp rx=0 tx=0).stop 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- 172.21.15.138:0/2966500640 shutdown_connections 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 --2- 172.21.15.138:0/2966500640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- 172.21.15.138:0/2966500640 >> 172.21.15.138:0/2966500640 conn(0x5558990df400 msgr2=0x5558991e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- 172.21.15.138:0/2966500640 shutdown_connections 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- 172.21.15.138:0/2966500640 wait complete. 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd24342880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555899103180 con 0x555899ed7800 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd21549640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd21549640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59258/0 (socket says 172.21.15.138:59258) 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.431+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 learned_addr learned my addr 172.21.15.138:0/2889096382 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555899150b60 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd21549640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x555899f9b9e0 tx=0x5558991f1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd19d3a640 1 -- 172.21.15.138:0/2889096382 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f7e40 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd19d3a640 1 -- 172.21.15.138:0/2889096382 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555899fb0480 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558991509c0 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd19d3a640 1 -- 172.21.15.138:0/2889096382 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5558990f74a0 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555899150820 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.432+0000 7fcd19d3a640 1 -- 172.21.15.138:0/2889096382 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5558991098c0 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd19d3a640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899470000 0x5558993c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd19d3a640 1 -- 172.21.15.138:0/2889096382 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5558990f1c00 con 0x555899ed7800 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd20547640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899470000 0x5558993c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed7c00 0x5558993c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470400 con 0x555899ed7c00 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd21549640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed7c00 0x5558993c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd20547640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899470000 0x5558993c1600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5558991eac00 tx=0x555899436120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd21549640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed7c00 0x5558993c1b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 20 realm 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.433+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470800 con 0x555899ed7c00 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470c00 con 0x555899ed7c00 2024-04-27T05:21:38.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471000 con 0x555899ed7c00 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899471400 0x5558993c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470400 con 0x555899471400 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd20d48640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899471400 0x5558993c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.434+0000 7fcd20d48640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899471400 0x5558993c2100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd24342880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd24342880 20 searching for the correct realm 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470800 con 0x555899471400 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd20d48640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899470800 0x5558993c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470400 con 0x555899470800 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.435+0000 7fcd20547640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899470800 0x5558993c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.436+0000 7fcd20547640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899470800 0x5558993c2c00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.436+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.436+0000 7fcd20547640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899470400 0x5558993c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.436+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.436+0000 7fcd21549640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899470400 0x5558993c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.436+0000 7fcd21549640 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899470400 0x5558993c3180 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.437+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470800 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.438+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470800 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.439+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.440+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470800 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.441+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470800 2024-04-27T05:21:38.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.442+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899471400 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470400 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899ed7c00 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ed6800 con 0x555899470800 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470c00 con 0x555899ed7c00 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.443+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471000 con 0x555899ed7c00 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471800 con 0x555899471400 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 20 zone default found 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 4 Realm: () 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 20 zonegroup default 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471c00 con 0x555899470400 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.444+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2000 con 0x555899471400 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2400 con 0x555899471400 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 started sync module instance, tier type = 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.445+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2800 con 0x555899470800 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.446+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.446+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93839016298752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2c00 con 0x555899470800 2024-04-27T05:21:38.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.446+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec3000 con 0x555899ed7c00 2024-04-27T05:21:38.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.447+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93839016298752] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.447+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.447+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93839016299904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec3400 con 0x555899ed7c00 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.447+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec3800 con 0x555899471400 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.448+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93839016299904] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.448+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.448+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93839016301056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2000 con 0x555899471400 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.448+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2400 con 0x555899470800 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.449+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.449+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93839016302208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec3800 con 0x555899470800 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.449+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93839016301056] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.449+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471800 con 0x555899ed7c00 2024-04-27T05:21:38.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.450+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.450+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93839016303360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899470c00 con 0x555899ed7c00 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.450+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471000 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.450+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93839016302208] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.451+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.451+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93839016303360] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.451+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93839016304512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471c00 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.451+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55589a034c00 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.452+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93839016304512] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.452+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.452+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93839016305664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2800 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.452+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2c00 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93839016306816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2400 con 0x555899470400 2024-04-27T05:21:38.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 20 add_watcher() i=0 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 20 add_watcher() i=1 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 20 add_watcher() i=2 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 20 add_watcher() i=3 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 20 add_watcher() i=4 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.453+0000 7fcd24342880 20 add_watcher() i=5 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.454+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93839016305664] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.454+0000 7fcd24342880 20 add_watcher() i=6 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.454+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93839016306816] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.454+0000 7fcd24342880 20 add_watcher() i=7 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.454+0000 7fcd24342880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.454+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec3800 con 0x555899ed7c00 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.455+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.455+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93839016307968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec3000 con 0x555899ed7c00 2024-04-27T05:21:38.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93839016307968] v29'127 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd07d16640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd24342880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd24342880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd24342880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd24342880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555899102780 con 0x555899ec3c00 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd20547640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd20547640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59266/0 (socket says 172.21.15.138:59266) 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.456+0000 7fcd20547640 1 -- 172.21.15.138:0/2253928210 learned_addr learned my addr 172.21.15.138:0/2253928210 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd20547640 1 -- 172.21.15.138:0/2253928210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555899150680 con 0x555899ec3c00 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd20547640 1 --2- 172.21.15.138:0/2253928210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55589a05c780 tx=0x5558994360c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9843ce743641b2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd07515640 1 -- 172.21.15.138:0/2253928210 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555899ec4580 con 0x555899ec3c00 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd07515640 1 -- 172.21.15.138:0/2253928210 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555899fb0000 con 0x555899ec3c00 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd07515640 1 -- 172.21.15.138:0/2253928210 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555899ec46e0 con 0x555899ec3c00 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd24342880 1 -- 172.21.15.138:0/2253928210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 msgr2=0x5558993c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.457+0000 7fcd24342880 1 --2- 172.21.15.138:0/2253928210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55589a05c780 tx=0x5558994360c0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 -- 172.21.15.138:0/2253928210 shutdown_connections 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 --2- 172.21.15.138:0/2253928210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 -- 172.21.15.138:0/2253928210 >> 172.21.15.138:0/2253928210 conn(0x555899ec3400 msgr2=0x5558991e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 -- 172.21.15.138:0/2253928210 shutdown_connections 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 -- 172.21.15.138:0/2253928210 wait complete. 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 Processor -- start 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 -- start start 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd24342880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x555899102500 con 0x555899ec3c00 2024-04-27T05:21:38.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd20547640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd20547640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59282/0 (socket says 172.21.15.138:59282) 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 learned_addr learned my addr 172.21.15.138:0/546020583 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.458+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558991504e0 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd20547640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x555899451740 tx=0x555899436000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd05d12640 1 -- 172.21.15.138:0/546020583 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555899ec4840 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd05d12640 1 -- 172.21.15.138:0/546020583 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555899fb0780 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd05d12640 1 -- 172.21.15.138:0/546020583 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555899ec4580 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558990e2ea0 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558990e36c0 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.459+0000 7fcd05d12640 1 -- 172.21.15.138:0/546020583 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x555899108b00 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd05d12640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899ec2800 0x5558993c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd05d12640 1 -- 172.21.15.138:0/546020583 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5558990f1dc0 con 0x555899ec3c00 2024-04-27T05:21:38.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd21549640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899ec2800 0x5558993c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed6800 0x55589951e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522000 con 0x555899ed6800 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd20d48640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed6800 0x55589951e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd21549640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899ec2800 0x5558993c3700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55589a05cba0 tx=0x555899436960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.460+0000 7fcd20d48640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed6800 0x55589951e000 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.461+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'49 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899ed6800 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.461+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899522400 0x55589951e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.461+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522800 con 0x555899522400 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.461+0000 7fcd20547640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899522400 0x55589951e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.461+0000 7fcd20547640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899522400 0x55589951e580 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.463+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.463+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899522c00 0x55589951eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.463+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523000 con 0x555899522c00 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.463+0000 7fcd21549640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899522c00 0x55589951eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.463+0000 7fcd21549640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899522c00 0x55589951eb00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.464+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.464+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523400 con 0x555899ed6800 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.465+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'128 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899ed6800 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.465+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899523800 0x55589951f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.465+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523c00 con 0x555899523800 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.465+0000 7fcd20d48640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899523800 0x55589951f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.465+0000 7fcd20d48640 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899523800 0x55589951f080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.466+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'41 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.466+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546000 con 0x555899522c00 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.467+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'38 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.467+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546400 con 0x555899ed6800 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.468+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'24 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899ed6800 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.468+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546800 con 0x555899522400 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.469+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'29 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.469+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546c00 con 0x555899523800 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.470+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.470+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547000 con 0x555899522c00 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.471+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.471+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547400 con 0x555899523800 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.472+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'42 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.472+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547800 con 0x555899522400 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.473+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.473+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547c00 con 0x555899523800 2024-04-27T05:21:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.474+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'21 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.474+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546400 con 0x555899522400 2024-04-27T05:21:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.475+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'46 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.475+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546c00 con 0x555899ed6800 2024-04-27T05:21:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.476+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'129 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899ed6800 2024-04-27T05:21:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.476+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522000 con 0x555899522c00 2024-04-27T05:21:38.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.477+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'35 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.477+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523400 con 0x555899523800 2024-04-27T05:21:38.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.478+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'43 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.478+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523c00 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.479+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.479+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2000 con 0x555899ed6800 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.480+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899ed6800 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.480+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523000 con 0x555899ed6800 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.481+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'130 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899ed6800 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.481+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522000 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.481+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'36 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.482+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523c00 con 0x555899523800 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.482+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'31 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.482+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546000 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.483+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.483+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547000 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.484+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'33 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.484+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2c00 con 0x555899522c00 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.485+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.485+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2400 con 0x555899522c00 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.486+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'39 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.486+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523400 con 0x555899522400 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.487+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'24 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.487+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523000 con 0x555899522c00 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.488+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.488+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523c00 con 0x555899522400 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.489+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'30 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.489+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2000 con 0x555899522c00 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.490+0000 7fcd21549640 1 -- 172.21.15.138:0/546020583 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'37 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482900 con 0x555899522c00 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.490+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547400 con 0x555899522400 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.491+0000 7fcd20547640 1 -- 172.21.15.138:0/546020583 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'47 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899483200 con 0x555899522400 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.491+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547c00 con 0x555899523800 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899523800 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd0350d640 20 reqs_thread_entry: start 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd24342880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd24342880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd01509640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd00ce7640 20 reqs_thread_entry: start 2024-04-27T05:21:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd24342880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd24342880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.492+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522800 con 0x555899470400 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523400 con 0x555899470400 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(63 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+15 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T05:21:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.493+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546800 con 0x555899470400 2024-04-27T05:21:38.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(64 p5YnriCv1nAtykxBrupQ [delete] v29'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd24342880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 93839016311424 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547800 con 0x555899471400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 93839016301056 notify 124554051590 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x555899ff56c0 con 0x555899471400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd1b53d640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051590 cookie 93839016301056 notifier 4573 bl.length()=170 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(65 notify.2 [notify cookie 93839016311424 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd1b53d640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546000 con 0x555899471400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 93839016311424 notify 124554051590 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555899ff5860 con 0x555899471400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd24342880 10 removing email index: 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.494+0000 7fcd24342880 10 removing user buckets index 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.495+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899546400 con 0x555899470400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.495+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(67 fud.buckets [delete] v29'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd24342880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93839016311424 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471c00 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== watch-notify(notify (1) cookie 93839016305664 notify 124554051599 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x5558994db860 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd1b53d640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051599 cookie 93839016305664 notifier 4573 bl.length()=160 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd1b53d640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899547000 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(68 notify.6 [notify cookie 93839016311424 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== watch-notify(notify_complete (2) cookie 93839016311424 notify 124554051599 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5558994dba00 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd24342880 10 removing user index: fud 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.496+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471000 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(70 fud [call,call,delete] v29'2 uv2 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd24342880 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-27T05:21:38.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd24342880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [notify cookie 93839016311424 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522000 con 0x555899470400 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== watch-notify(notify (1) cookie 93839016304512 notify 124554051600 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x5558994dba00 con 0x555899470400 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(71 notify.5 [notify cookie 93839016311424 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd1ad3c640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051600 cookie 93839016304512 notifier 4573 bl.length()=152 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd1ad3c640 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5558995b4000 con 0x555899470400 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.497+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(72 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.498+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== watch-notify(notify_complete (2) cookie 93839016311424 notify 124554051600 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5558994da000 con 0x555899470400 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.498+0000 7fcd07d16640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.498+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93839016307968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523000 con 0x555899ed7c00 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.499+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 93839016307968] v29'131 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.499+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93839016298752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522800 con 0x555899470800 2024-04-27T05:21:38.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.500+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 93839016298752] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.500+0000 7fcd24342880 20 remove_watcher() i=0 2024-04-27T05:21:38.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.500+0000 7fcd24342880 2 removed watcher, disabling cache 2024-04-27T05:21:38.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.500+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93839016299904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2c00 con 0x555899ed7c00 2024-04-27T05:21:38.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.501+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 93839016299904] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.501+0000 7fcd24342880 20 remove_watcher() i=1 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.501+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93839016301056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2400 con 0x555899471400 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.502+0000 7fcd20d48640 1 -- 172.21.15.138:0/2889096382 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 93839016301056] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482fc0 con 0x555899471400 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.502+0000 7fcd24342880 20 remove_watcher() i=2 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.502+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93839016302208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899ec2000 con 0x555899470800 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.503+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 93839016302208] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899483200 con 0x555899470800 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.503+0000 7fcd24342880 20 remove_watcher() i=3 2024-04-27T05:21:38.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.503+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93839016303360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523400 con 0x555899ed7c00 2024-04-27T05:21:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.504+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 93839016303360] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899ed7c00 2024-04-27T05:21:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.504+0000 7fcd24342880 20 remove_watcher() i=4 2024-04-27T05:21:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.504+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93839016304512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899522000 con 0x555899470400 2024-04-27T05:21:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.505+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 93839016304512] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.505+0000 7fcd24342880 20 remove_watcher() i=5 2024-04-27T05:21:38.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.505+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93839016305664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899523000 con 0x555899470400 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.506+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 93839016305664] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.506+0000 7fcd24342880 20 remove_watcher() i=6 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.506+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93839016306816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555899471c00 con 0x555899470400 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd21549640 1 -- 172.21.15.138:0/2889096382 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 93839016306816] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555899482900 con 0x555899470400 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 20 remove_watcher() i=7 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899470400 msgr2=0x5558993c3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899470400 0x5558993c3180 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed7c00 msgr2=0x5558993c1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed7c00 0x5558993c1b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899470800 msgr2=0x5558993c2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899470800 0x5558993c2c00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899471400 msgr2=0x5558993c2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899471400 0x5558993c2100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899470000 msgr2=0x5558993c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899470000 0x5558993c1600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x5558991eac00 tx=0x555899436120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 msgr2=0x5558993c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 --2- 172.21.15.138:0/2889096382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ed7800 0x5558993c0b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x555899f9b9e0 tx=0x5558991f1a40 comp rx=0 tx=0).stop 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 reap_dead start 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd20547640 1 -- 172.21.15.138:0/2889096382 reap_dead start 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 shutdown_connections 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 >> 172.21.15.138:0/2889096382 conn(0x5558990df400 msgr2=0x5558991e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 shutdown_connections 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.507+0000 7fcd24342880 1 -- 172.21.15.138:0/2889096382 wait complete. 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899522400 msgr2=0x55589951e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x555899522400 0x55589951e580 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed6800 msgr2=0x55589951e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x555899ed6800 0x55589951e000 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899523800 msgr2=0x55589951f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x555899523800 0x55589951f080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899522c00 msgr2=0x55589951eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x555899522c00 0x55589951eb00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899ec2800 msgr2=0x5558993c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x555899ec2800 0x5558993c3700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55589a05cba0 tx=0x555899436960 comp rx=0 tx=0).stop 2024-04-27T05:21:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 msgr2=0x5558993c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/546020583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555899ec3c00 0x5558993c2680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x555899451740 tx=0x555899436000 comp rx=0 tx=0).stop 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd20d48640 1 -- 172.21.15.138:0/546020583 reap_dead start 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 shutdown_connections 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 >> 172.21.15.138:0/546020583 conn(0x555899ec3400 msgr2=0x5558991e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 shutdown_connections 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/546020583 wait complete. 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55589a034000 msgr2=0x5558993c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55589a034000 0x5558993c1080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x555899fe2450 tx=0x5558991f14a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 msgr2=0x5558993c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x555899f9af00 tx=0x5558991f0c60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 shutdown_connections 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55589a034000 0x5558993c1080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 --2- 172.21.15.138:0/527656535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5558990df000 0x5558993c0580 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 >> 172.21.15.138:0/527656535 conn(0x5558990dec00 msgr2=0x5558991e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 shutdown_connections 2024-04-27T05:21:38.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.508+0000 7fcd24342880 1 -- 172.21.15.138:0/527656535 wait complete. 2024-04-27T05:21:38.517 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T05:21:38.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T05:21:38.517 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-27T05:21:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.609 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf71722880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf71722880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56498d0d6600 con 0x56498cdff400 2024-04-27T05:21:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf71722880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x56498ce3e280 con 0x56498cdff000 2024-04-27T05:21:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf6dfcd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf6d7cc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x56498cdff400 0x56498cf06800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42846/0 (socket says 172.21.15.138:42846) 2024-04-27T05:21:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.607+0000 7faf6d7cc640 1 -- 172.21.15.138:0/193514484 learned_addr learned my addr 172.21.15.138:0/193514484 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.608+0000 7faf6dfcd640 1 -- 172.21.15.138:0/193514484 >> v1:172.21.15.114:6789/0 conn(0x56498cdff400 legacy=0x56498cf06800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.608+0000 7faf6cfcb640 1 -- 172.21.15.138:0/193514484 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979615559 0 0) 0x56498ce029c0 con 0x56498cdff400 2024-04-27T05:21:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.608+0000 7faf6dfcd640 1 -- 172.21.15.138:0/193514484 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce03380 con 0x56498cdff000 2024-04-27T05:21:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.609+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/193514484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x56498cf0ad50 tx=0x56498cf10de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae879d5158b6ee46 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.609+0000 7faf6cfcb640 1 -- 172.21.15.138:0/193514484 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce17ce0 con 0x56498cdff000 2024-04-27T05:21:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.609+0000 7faf6cfcb640 1 -- 172.21.15.138:0/193514484 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498d0d7b00 con 0x56498cdff000 2024-04-27T05:21:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.609+0000 7faf6cfcb640 1 -- 172.21.15.138:0/193514484 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce17e40 con 0x56498cdff000 2024-04-27T05:21:38.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 -- 172.21.15.138:0/193514484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 msgr2=0x56498d0e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 --2- 172.21.15.138:0/193514484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x56498cf0ad50 tx=0x56498cf10de0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 -- 172.21.15.138:0/193514484 shutdown_connections 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 --2- 172.21.15.138:0/193514484 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 -- 172.21.15.138:0/193514484 >> 172.21.15.138:0/193514484 conn(0x56498cdfec00 msgr2=0x56498cf06000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 -- 172.21.15.138:0/193514484 shutdown_connections 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.610+0000 7faf71722880 1 -- 172.21.15.138:0/193514484 wait complete. 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf71722880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf71722880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498ce3e780 con 0x56498cdff000 2024-04-27T05:21:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6dfcd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6dfcd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59296/0 (socket says 172.21.15.138:59296) 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6dfcd640 1 -- 172.21.15.138:0/3121906702 learned_addr learned my addr 172.21.15.138:0/3121906702 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6dfcd640 1 -- 172.21.15.138:0/3121906702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce02ea0 con 0x56498cdff000 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/3121906702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x56498dc05590 tx=0x56498cf10cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d0647c5fc0dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6c7ca640 1 -- 172.21.15.138:0/3121906702 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce174a0 con 0x56498cdff000 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6c7ca640 1 -- 172.21.15.138:0/3121906702 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498d0d7e00 con 0x56498cdff000 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.611+0000 7faf6c7ca640 1 -- 172.21.15.138:0/3121906702 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce17ce0 con 0x56498cdff000 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- 172.21.15.138:0/3121906702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 msgr2=0x56498d0e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 --2- 172.21.15.138:0/3121906702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x56498dc05590 tx=0x56498cf10cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- 172.21.15.138:0/3121906702 shutdown_connections 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 --2- 172.21.15.138:0/3121906702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- 172.21.15.138:0/3121906702 >> 172.21.15.138:0/3121906702 conn(0x56498cdfec00 msgr2=0x56498cf07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- 172.21.15.138:0/3121906702 shutdown_connections 2024-04-27T05:21:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- 172.21.15.138:0/3121906702 wait complete. 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf71722880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498ce3e8c0 con 0x56498cdff000 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf6dfcd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf6dfcd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59312/0 (socket says 172.21.15.138:59312) 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.612+0000 7faf6dfcd640 1 -- 172.21.15.138:0/4067144309 learned_addr learned my addr 172.21.15.138:0/4067144309 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf6dfcd640 1 -- 172.21.15.138:0/4067144309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce036c0 con 0x56498cdff000 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x56498dcbae10 tx=0x56498cf10c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf6afc7640 1 -- 172.21.15.138:0/4067144309 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce17e40 con 0x56498cdff000 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf6afc7640 1 -- 172.21.15.138:0/4067144309 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498dcbc000 con 0x56498cdff000 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56498ce031e0 con 0x56498cdff000 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf6afc7640 1 -- 172.21.15.138:0/4067144309 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce174a0 con 0x56498cdff000 2024-04-27T05:21:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56498ce70ea0 con 0x56498cdff000 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.613+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf71722880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf71722880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498ce22280 con 0x56498dbf6c00 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6afc7640 1 -- 172.21.15.138:0/4067144309 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56498ce28dc0 con 0x56498cdff000 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6e7ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6e7ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59328/0 (socket says 172.21.15.138:59328) 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2036623911 learned_addr learned my addr 172.21.15.138:0/2036623911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6afc7640 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dbf7c00 0x56498d0e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6afc7640 1 -- 172.21.15.138:0/4067144309 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56498ce11a40 con 0x56498cdff000 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dbf7c00 0x56498d0e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2036623911 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce70d00 con 0x56498dbf6c00 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/2036623911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x56498dd03590 tx=0x56498cf11980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996a3abc34c5d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dbf7c00 0x56498d0e1080 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56498dd022d0 tx=0x56498cf10cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf687c2640 1 -- 172.21.15.138:0/2036623911 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce17e40 con 0x56498dbf6c00 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf687c2640 1 -- 172.21.15.138:0/2036623911 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498dcbc600 con 0x56498dbf6c00 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.614+0000 7faf687c2640 1 -- 172.21.15.138:0/2036623911 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce174a0 con 0x56498dbf6c00 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- 172.21.15.138:0/2036623911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 msgr2=0x56498d0e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 --2- 172.21.15.138:0/2036623911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x56498dd03590 tx=0x56498cf11980 comp rx=0 tx=0).stop 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- 172.21.15.138:0/2036623911 shutdown_connections 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 --2- 172.21.15.138:0/2036623911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- 172.21.15.138:0/2036623911 >> 172.21.15.138:0/2036623911 conn(0x56498cdff400 msgr2=0x56498cf09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- 172.21.15.138:0/2036623911 shutdown_connections 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- 172.21.15.138:0/2036623911 wait complete. 2024-04-27T05:21:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf71722880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498ce23180 con 0x56498dbf6c00 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf6e7ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf6e7ce640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59334/0 (socket says 172.21.15.138:59334) 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.615+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 learned_addr learned my addr 172.21.15.138:0/2040223359 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce70b60 con 0x56498dbf6c00 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x56498dcbb410 tx=0x56498cf119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf66fbf640 1 -- 172.21.15.138:0/2040223359 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce17e40 con 0x56498dbf6c00 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf66fbf640 1 -- 172.21.15.138:0/2040223359 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498dcbc780 con 0x56498dbf6c00 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56498ce709c0 con 0x56498dbf6c00 2024-04-27T05:21:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf66fbf640 1 -- 172.21.15.138:0/2040223359 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498ce174a0 con 0x56498dbf6c00 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.616+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56498ce70820 con 0x56498dbf6c00 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf66fbf640 1 -- 172.21.15.138:0/2040223359 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56498ce28b00 con 0x56498dbf6c00 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf66fbf640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498d17c000 0x56498d0e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf66fbf640 1 -- 172.21.15.138:0/2040223359 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56498ce11c00 con 0x56498dbf6c00 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56498dbf7000 0x56498d0e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498d17c000 0x56498d0e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dbf7400 con 0x56498dbf7000 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56498dbf7000 0x56498d0e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498d17c000 0x56498d0e1600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x56498cf0b410 tx=0x56498d144120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.617+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56498dbf7000 0x56498d0e1b80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 realm 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dbf7800 con 0x56498dbf7000 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17d000 con 0x56498dbf7000 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17d400 con 0x56498dbf7000 2024-04-27T05:21:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56498d17d800 0x56498d0e2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17dc00 con 0x56498d17d800 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.618+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56498d17d800 0x56498d0e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56498d17d800 0x56498d0e2100 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf71722880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf71722880 20 searching for the correct realm 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17d000 con 0x56498d17d800 2024-04-27T05:21:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf6dfcd640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56498d17d000 0x56498d0e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.619+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17dc00 con 0x56498d17d000 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56498d17d000 0x56498d0e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56498d17d000 0x56498d0e2c00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56498d17dc00 0x56498d0e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56498d17dc00 0x56498d0e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.620+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56498d17dc00 0x56498d0e3180 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.621+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d000 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.622+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d000 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.623+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.624+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d000 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.625+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.626+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d000 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d800 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17dc00 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498dbf7000 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.627+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17cc00 con 0x56498d17d000 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17d400 con 0x56498dbf7000 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dbf7400 con 0x56498dbf7000 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dbf7800 con 0x56498d17d800 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 zone default found 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 4 Realm: () 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 zonegroup default 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.628+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6000 con 0x56498d17dc00 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6400 con 0x56498d17d800 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6800 con 0x56498d17d800 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 started sync module instance, tier type = 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.629+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6c00 con 0x56498d17d000 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.630+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.630+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94873899075840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7000 con 0x56498d17d000 2024-04-27T05:21:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.630+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7400 con 0x56498dbf7000 2024-04-27T05:21:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.631+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94873899075840] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.631+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.631+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94873899076992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7800 con 0x56498dbf7000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.631+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7c00 con 0x56498d17d800 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.632+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94873899076992] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.632+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.632+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94873899078144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6400 con 0x56498d17d800 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.632+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6800 con 0x56498d17d000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.633+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94873899078144] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.633+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.633+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94873899079296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7c00 con 0x56498d17d000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.633+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6000 con 0x56498dbf7000 2024-04-27T05:21:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.634+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94873899079296] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.634+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.635+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94873899080448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dbf7800 con 0x56498dbf7000 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.635+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dbf7400 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.635+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.635+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94873899081600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d17d400 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.635+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94873899080448] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.635+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dd5f800 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.636+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94873899081600] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.638+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.638+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94873899082752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498dd5f400 con 0x56498d17dc00 2024-04-27T05:21:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.638+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6c00 con 0x56498d17dc00 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94873899082752] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94873899083904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7000 con 0x56498d17dc00 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=0 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=1 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=2 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=3 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=4 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=5 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.639+0000 7faf71722880 20 add_watcher() i=6 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.640+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94873899083904] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.640+0000 7faf71722880 20 add_watcher() i=7 2024-04-27T05:21:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.640+0000 7faf71722880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.640+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a6800 con 0x56498dbf7000 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.640+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.640+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94873899085056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d1a7400 con 0x56498dbf7000 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.641+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94873899085056] v29'132 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.641+0000 7faf54f9b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.641+0000 7faf71722880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.641+0000 7faf71722880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.641+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.641+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf71722880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf71722880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498ce22780 con 0x56498d17cc00 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf6d7cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf6d7cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59336/0 (socket says 172.21.15.138:59336) 2024-04-27T05:21:38.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf6d7cc640 1 -- 172.21.15.138:0/4232635999 learned_addr learned my addr 172.21.15.138:0/4232635999 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf6d7cc640 1 -- 172.21.15.138:0/4232635999 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce70680 con 0x56498d17cc00 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/4232635999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x56498d161620 tx=0x56498d144180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b6defde6fe386 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf5479a640 1 -- 172.21.15.138:0/4232635999 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498d1a8580 con 0x56498d17cc00 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf5479a640 1 -- 172.21.15.138:0/4232635999 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498dcbc000 con 0x56498d17cc00 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.642+0000 7faf5479a640 1 -- 172.21.15.138:0/4232635999 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498d1a86e0 con 0x56498d17cc00 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 -- 172.21.15.138:0/4232635999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 msgr2=0x56498d0e2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 --2- 172.21.15.138:0/4232635999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x56498d161620 tx=0x56498d144180 comp rx=0 tx=0).stop 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 -- 172.21.15.138:0/4232635999 shutdown_connections 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 --2- 172.21.15.138:0/4232635999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 -- 172.21.15.138:0/4232635999 >> 172.21.15.138:0/4232635999 conn(0x56498d1a7800 msgr2=0x56498cf09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 -- 172.21.15.138:0/4232635999 shutdown_connections 2024-04-27T05:21:38.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 -- 172.21.15.138:0/4232635999 wait complete. 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 Processor -- start 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.643+0000 7faf71722880 1 -- start start 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf71722880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf71722880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498ce22500 con 0x56498d17cc00 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf6d7cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf6d7cc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59344/0 (socket says 172.21.15.138:59344) 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf6d7cc640 1 -- 172.21.15.138:0/911846270 learned_addr learned my addr 172.21.15.138:0/911846270 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf6d7cc640 1 -- 172.21.15.138:0/911846270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498ce704e0 con 0x56498d17cc00 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf6d7cc640 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x56498d161830 tx=0x56498d144ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf52f97640 1 -- 172.21.15.138:0/911846270 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498d1a8840 con 0x56498d17cc00 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf52f97640 1 -- 172.21.15.138:0/911846270 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56498dcbca80 con 0x56498d17cc00 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56498ce03380 con 0x56498d17cc00 2024-04-27T05:21:38.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf52f97640 1 -- 172.21.15.138:0/911846270 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56498d1a8580 con 0x56498d17cc00 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.644+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56498ce02ea0 con 0x56498d17cc00 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf52f97640 1 -- 172.21.15.138:0/911846270 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56498ce298c0 con 0x56498d17cc00 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf52f97640 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dd5f800 0x56498d0e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf52f97640 1 -- 172.21.15.138:0/911846270 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56498ce11dc0 con 0x56498d17cc00 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf71722880 5 note: GC not initialized 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dd5f800 0x56498d0e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf50792640 20 reqs_thread_entry: start 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.645+0000 7faf71722880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf6e7ce640 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dd5f800 0x56498d0e3700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56498d161170 tx=0x56498d1448a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf4e78e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:08 2024 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf4df6c640 20 reqs_thread_entry: start 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d276000 con 0x56498d17dc00 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 246+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d276400 con 0x56498d17dc00 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.646+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~385 in=385b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d276800 con 0x56498d17dc00 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.647+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~385] v29'5 uv5 ondisk = 0) v8 ==== 288+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.647+0000 7faf71722880 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.647+0000 7faf71722880 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.647+0000 7faf71722880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.647+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94873899087360 in=588b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d276c00 con 0x56498d17d000 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== watch-notify(notify (1) cookie 94873899079296 notify 124554051596 ret 0) v3 ==== 618+0+0 (crc 0 0 0) 0x56498d1ffd40 con 0x56498d17d000 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.3 [notify cookie 94873899087360 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf67fc1640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051596 cookie 94873899079296 notifier 4581 bl.length()=576 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf67fc1640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf67fc1640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf67fc1640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d286000 con 0x56498d17d000 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== watch-notify(notify_complete (2) cookie 94873899087360 notify 124554051596 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56498d1ffa00 con 0x56498d17d000 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.648+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d277000 con 0x56498d17dc00 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v29'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf71722880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf71722880 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf71722880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:21:38.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94873899087360 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d277400 con 0x56498d17d800 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94873899078144 notify 124554051591 ret 0) v3 ==== 234+0+0 (crc 0 0 0) 0x56498dd06d00 con 0x56498d17d800 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf67fc1640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051591 cookie 94873899078144 notifier 4581 bl.length()=192 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf67fc1640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf67fc1640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [notify cookie 94873899087360 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.649+0000 7faf67fc1640 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d286400 con 0x56498d17d800 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.650+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== watch-notify(notify_complete (2) cookie 94873899087360 notify 124554051591 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56498dd06ea0 con 0x56498d17d800 2024-04-27T05:21:38.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.650+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "testx$tenanteduser", 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "tenanted-user", 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "", 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 1, 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "testx$tenanteduser", 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:38.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.650+0000 7faf54f9b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.650+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94873899085056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d277800 con 0x56498dbf7000 2024-04-27T05:21:38.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.651+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94873899085056] v29'133 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.651+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94873899075840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d277c00 con 0x56498d17d000 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.652+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94873899075840] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.652+0000 7faf71722880 20 remove_watcher() i=0 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.652+0000 7faf71722880 2 removed watcher, disabling cache 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.652+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94873899076992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d286c00 con 0x56498dbf7000 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.653+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94873899076992] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.653+0000 7faf71722880 20 remove_watcher() i=1 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.653+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94873899078144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d287000 con 0x56498d17d800 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.654+0000 7faf6dfcd640 1 -- 172.21.15.138:0/2040223359 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94873899078144] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194fc0 con 0x56498d17d800 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.654+0000 7faf71722880 20 remove_watcher() i=2 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.654+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94873899079296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d287400 con 0x56498d17d000 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.655+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94873899079296] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d195200 con 0x56498d17d000 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.655+0000 7faf71722880 20 remove_watcher() i=3 2024-04-27T05:21:38.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.655+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94873899080448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d287800 con 0x56498dbf7000 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.656+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94873899080448] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498dbf7000 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.656+0000 7faf71722880 20 remove_watcher() i=4 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.656+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94873899081600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d287c00 con 0x56498d17dc00 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.657+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94873899081600] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.657+0000 7faf71722880 20 remove_watcher() i=5 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.657+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94873899082752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d276000 con 0x56498d17dc00 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.658+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94873899082752] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.658+0000 7faf71722880 20 remove_watcher() i=6 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.658+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94873899083904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56498d276400 con 0x56498d17dc00 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf6e7ce640 1 -- 172.21.15.138:0/2040223359 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94873899083904] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56498d194900 con 0x56498d17dc00 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 20 remove_watcher() i=7 2024-04-27T05:21:38.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56498d17dc00 msgr2=0x56498d0e3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56498d17dc00 0x56498d0e3180 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56498dbf7000 msgr2=0x56498d0e1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56498dbf7000 0x56498d0e1b80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56498d17d000 msgr2=0x56498d0e2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56498d17d000 0x56498d0e2c00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56498d17d800 msgr2=0x56498d0e2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56498d17d800 0x56498d0e2100 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498d17c000 msgr2=0x56498d0e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498d17c000 0x56498d0e1600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x56498cf0b410 tx=0x56498d144120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 msgr2=0x56498d0e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x56498dcbb410 tx=0x56498cf119e0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf6d7cc640 1 -- 172.21.15.138:0/2040223359 reap_dead start 2024-04-27T05:21:38.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 shutdown_connections 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 --2- 172.21.15.138:0/2040223359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498dbf6c00 0x56498d0e0b00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 >> 172.21.15.138:0/2040223359 conn(0x56498cdff400 msgr2=0x56498cf09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 shutdown_connections 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.659+0000 7faf71722880 1 -- 172.21.15.138:0/2040223359 wait complete. 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dd5f800 msgr2=0x56498d0e3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dd5f800 0x56498d0e3700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56498d161170 tx=0x56498d1448a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 msgr2=0x56498d0e2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x56498d161830 tx=0x56498d144ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 shutdown_connections 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dd5f800 0x56498d0e3700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/911846270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498d17cc00 0x56498d0e2680 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 >> 172.21.15.138:0/911846270 conn(0x56498d1a7800 msgr2=0x56498cf09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 shutdown_connections 2024-04-27T05:21:38.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/911846270 wait complete. 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dbf7c00 msgr2=0x56498d0e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dbf7c00 0x56498d0e1080 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56498dd022d0 tx=0x56498cf10cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 msgr2=0x56498d0e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x56498dcbae10 tx=0x56498cf10c60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 shutdown_connections 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56498dbf7c00 0x56498d0e1080 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 --2- 172.21.15.138:0/4067144309 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56498cdff000 0x56498d0e0580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 >> 172.21.15.138:0/4067144309 conn(0x56498cdfec00 msgr2=0x56498cf07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 shutdown_connections 2024-04-27T05:21:38.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.660+0000 7faf71722880 1 -- 172.21.15.138:0/4067144309 wait complete. 2024-04-27T05:21:38.688 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:38.689 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T05:21:38.689 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T05:21:38.689 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-04-27T05:21:38.735 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.735 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.733+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.733+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.733+0000 7fd4f7c0f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55806d6ad400 0x55806d98e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.733+0000 7fd4f7c0f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55806d984600 con 0x55806d6ad000 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f7c0f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55806d6ec280 con 0x55806d6ad400 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f38db640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55806d6ad400 0x55806d98e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f40dc640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55806d6ad000 0x55806d7b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42848/0 (socket says 172.21.15.138:42848) 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f40dc640 1 -- 172.21.15.138:0/2351817594 learned_addr learned my addr 172.21.15.138:0/2351817594 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f38db640 1 -- 172.21.15.138:0/2351817594 >> v1:172.21.15.114:6789/0 conn(0x55806d6ad000 legacy=0x55806d7b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f38db640 1 -- 172.21.15.138:0/2351817594 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d6b1380 con 0x55806d6ad400 2024-04-27T05:21:38.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.734+0000 7fd4f30da640 1 -- 172.21.15.138:0/2351817594 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337414192 0 0) 0x55806d6b09c0 con 0x55806d6ad000 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f38db640 1 --2- 172.21.15.138:0/2351817594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad400 0x55806d98e580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55806d7b8c00 tx=0x55806d7bede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c03ec773b4a258 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f30da640 1 -- 172.21.15.138:0/2351817594 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c5ce0 con 0x55806d6ad400 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f30da640 1 -- 172.21.15.138:0/2351817594 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806d985b00 con 0x55806d6ad400 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f30da640 1 -- 172.21.15.138:0/2351817594 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c5e40 con 0x55806d6ad400 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2351817594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad400 msgr2=0x55806d98e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/2351817594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad400 0x55806d98e580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55806d7b8c00 tx=0x55806d7bede0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2351817594 shutdown_connections 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/2351817594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad400 0x55806d98e580 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2351817594 >> 172.21.15.138:0/2351817594 conn(0x55806d6acc00 msgr2=0x55806d7b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2351817594 shutdown_connections 2024-04-27T05:21:38.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.736+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2351817594 wait complete. 2024-04-27T05:21:38.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f7c0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f7c0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d6ec780 con 0x55806d6ad000 2024-04-27T05:21:38.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f40dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f40dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59356/0 (socket says 172.21.15.138:59356) 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.737+0000 7fd4f40dc640 1 -- 172.21.15.138:0/2134541368 learned_addr learned my addr 172.21.15.138:0/2134541368 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f40dc640 1 -- 172.21.15.138:0/2134541368 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d6b0ea0 con 0x55806d6ad000 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/2134541368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55806e568c90 tx=0x55806d7becc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20ffed1a808e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f28d9640 1 -- 172.21.15.138:0/2134541368 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c54a0 con 0x55806d6ad000 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f28d9640 1 -- 172.21.15.138:0/2134541368 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806d985e00 con 0x55806d6ad000 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f28d9640 1 -- 172.21.15.138:0/2134541368 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c5ce0 con 0x55806d6ad000 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2134541368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 msgr2=0x55806d98e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/2134541368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55806e568c90 tx=0x55806d7becc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2134541368 shutdown_connections 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/2134541368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2134541368 >> 172.21.15.138:0/2134541368 conn(0x55806d6acc00 msgr2=0x55806d7b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2134541368 shutdown_connections 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.738+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/2134541368 wait complete. 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f7c0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f7c0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d6ec8c0 con 0x55806d6ad000 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f40dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f40dc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59358/0 (socket says 172.21.15.138:59358) 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3381747822 learned_addr learned my addr 172.21.15.138:0/3381747822 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3381747822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d6b16c0 con 0x55806d6ad000 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.739+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55806e569350 tx=0x55806d7bec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f10d6640 1 -- 172.21.15.138:0/3381747822 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c5e40 con 0x55806d6ad000 2024-04-27T05:21:38.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55806d6b11e0 con 0x55806d6ad000 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f10d6640 1 -- 172.21.15.138:0/3381747822 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806e57e000 con 0x55806d6ad000 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f10d6640 1 -- 172.21.15.138:0/3381747822 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c54a0 con 0x55806d6ad000 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806d71eea0 con 0x55806d6ad000 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f7c0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f7c0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d6d0280 con 0x55806e4a5800 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f10d6640 1 -- 172.21.15.138:0/3381747822 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55806d6d7600 con 0x55806d6ad000 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f10d6640 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e602000 0x55806d98f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f10d6640 1 -- 172.21.15.138:0/3381747822 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55806d6bfa40 con 0x55806d6ad000 2024-04-27T05:21:38.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f48dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f48dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59368/0 (socket says 172.21.15.138:59368) 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.740+0000 7fd4f48dd640 1 -- 172.21.15.138:0/768468238 learned_addr learned my addr 172.21.15.138:0/768468238 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e602000 0x55806d98f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f48dd640 1 -- 172.21.15.138:0/768468238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d71ed00 con 0x55806e4a5800 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/768468238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55806e569e90 tx=0x55806d7bec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688ba42f2d633611 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e602000 0x55806d98f080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55806e5b2420 tx=0x55806d7bf4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4ee8d1640 1 -- 172.21.15.138:0/768468238 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c5e40 con 0x55806e4a5800 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4ee8d1640 1 -- 172.21.15.138:0/768468238 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806e57e300 con 0x55806e4a5800 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4ee8d1640 1 -- 172.21.15.138:0/768468238 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c54a0 con 0x55806e4a5800 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/768468238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 msgr2=0x55806d98eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/768468238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55806e569e90 tx=0x55806d7bec00 comp rx=0 tx=0).stop 2024-04-27T05:21:38.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/768468238 shutdown_connections 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/768468238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/768468238 >> 172.21.15.138:0/768468238 conn(0x55806d6ad400 msgr2=0x55806d7b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/768468238 shutdown_connections 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.741+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/768468238 wait complete. 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f7c0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f7c0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d6d1180 con 0x55806e4a5800 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f48dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f48dd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59384/0 (socket says 172.21.15.138:59384) 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 learned_addr learned my addr 172.21.15.138:0/3897756181 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.742+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d71eb60 con 0x55806e4a5800 2024-04-27T05:21:38.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55806e569950 tx=0x55806d7bf1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4ed0ce640 1 -- 172.21.15.138:0/3897756181 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c5e40 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4ed0ce640 1 -- 172.21.15.138:0/3897756181 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806e57e480 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4ed0ce640 1 -- 172.21.15.138:0/3897756181 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806d6c54a0 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55806d71e9c0 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806d71e820 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4ed0ce640 1 -- 172.21.15.138:0/3897756181 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55806d6d78c0 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.743+0000 7fd4ed0ce640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806da32000 0x55806d98f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4ed0ce640 1 -- 172.21.15.138:0/3897756181 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55806d6bfc00 con 0x55806e4a5800 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f38db640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806da32000 0x55806d98f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55806e4a5c00 0x55806d98fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32400 con 0x55806e4a5c00 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55806e4a5c00 0x55806d98fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f38db640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806da32000 0x55806d98f600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55806d7b8c00 tx=0x55806d9f8120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55806e4a5c00 0x55806d98fb80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 20 realm 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32800 con 0x55806e4a5c00 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.744+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32c00 con 0x55806e4a5c00 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33000 con 0x55806e4a5c00 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55806da33400 0x55806d990100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32400 con 0x55806da33400 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55806da33400 0x55806d990100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.745+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55806da33400 0x55806d990100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f7c0f880 20 searching for the correct realm 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32800 con 0x55806da33400 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f40dc640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55806da32800 0x55806d990c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32400 con 0x55806da32800 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f38db640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55806da32800 0x55806d990c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.746+0000 7fd4f38db640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55806da32800 0x55806d990c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f38db640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55806da32400 0x55806d991180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55806da32400 0x55806d991180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55806da32400 0x55806d991180 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.747+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32800 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.748+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32800 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.749+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.750+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.751+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32800 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.752+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32800 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da33400 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32400 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.753+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806e4a5c00 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e4a4800 con 0x55806da32800 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32c00 con 0x55806e4a5c00 2024-04-27T05:21:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33000 con 0x55806e4a5c00 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33800 con 0x55806da33400 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.754+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 zone default found 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 4 Realm: () 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 zonegroup default 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33c00 con 0x55806da32400 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490000 con 0x55806da33400 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490400 con 0x55806da33400 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 started sync module instance, tier type = 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.755+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490800 con 0x55806da32800 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.756+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.756+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94010082879744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490c00 con 0x55806da32800 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.756+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e491000 con 0x55806e4a5c00 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.757+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94010082879744] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.758+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.758+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94010082880896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e491400 con 0x55806e4a5c00 2024-04-27T05:21:38.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.758+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e491800 con 0x55806da33400 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.759+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94010082880896] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.759+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.759+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94010082882048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490000 con 0x55806da33400 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.759+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490400 con 0x55806da32800 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.760+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.760+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94010082882048] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.760+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94010082883200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e491800 con 0x55806da32800 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.760+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33800 con 0x55806e4a5c00 2024-04-27T05:21:38.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.761+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.761+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94010082884352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da32c00 con 0x55806e4a5c00 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.761+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33000 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.761+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94010082883200] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.762+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94010082884352] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.762+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.762+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94010082885504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33c00 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.762+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e603800 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.763+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94010082885504] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.763+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.763+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94010082886656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490800 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.763+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490c00 con 0x55806da32400 2024-04-27T05:21:38.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94010082887808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490400 con 0x55806da32400 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94010082886656] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=0 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=1 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=2 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=3 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=4 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=5 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.764+0000 7fd4f7c0f880 20 add_watcher() i=6 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.765+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94010082887808] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.765+0000 7fd4f7c0f880 20 add_watcher() i=7 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.765+0000 7fd4f7c0f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.765+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e491800 con 0x55806e4a5c00 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.765+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.765+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94010082888960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e491000 con 0x55806e4a5c00 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94010082888960] v29'134 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4db0aa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f7c0f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f7c0f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f7c0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.766+0000 7fd4f7c0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d6d0780 con 0x55806e491c00 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4f38db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4f38db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59388/0 (socket says 172.21.15.138:59388) 2024-04-27T05:21:38.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4f38db640 1 -- 172.21.15.138:0/4064117313 learned_addr learned my addr 172.21.15.138:0/4064117313 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4f38db640 1 -- 172.21.15.138:0/4064117313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d71e680 con 0x55806e491c00 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4f38db640 1 --2- 172.21.15.138:0/4064117313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55806da12c30 tx=0x55806d9f80c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e15a4d825b24ab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4da8a9640 1 -- 172.21.15.138:0/4064117313 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806e492580 con 0x55806e491c00 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4da8a9640 1 -- 172.21.15.138:0/4064117313 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806e57e000 con 0x55806e491c00 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.767+0000 7fd4da8a9640 1 -- 172.21.15.138:0/4064117313 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806e4926e0 con 0x55806e491c00 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/4064117313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 msgr2=0x55806d990680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/4064117313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55806da12c30 tx=0x55806d9f80c0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/4064117313 shutdown_connections 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/4064117313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/4064117313 >> 172.21.15.138:0/4064117313 conn(0x55806e491400 msgr2=0x55806d7b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/4064117313 shutdown_connections 2024-04-27T05:21:38.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/4064117313 wait complete. 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 Processor -- start 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- start start 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.768+0000 7fd4f7c0f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806d6d0500 con 0x55806e491c00 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f38db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f38db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59402/0 (socket says 172.21.15.138:59402) 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f38db640 1 -- 172.21.15.138:0/845459743 learned_addr learned my addr 172.21.15.138:0/845459743 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f38db640 1 -- 172.21.15.138:0/845459743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55806d6b0ea0 con 0x55806e491c00 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f38db640 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55806da137a0 tx=0x55806d9f8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4d90a6640 1 -- 172.21.15.138:0/845459743 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806e492840 con 0x55806e491c00 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4d90a6640 1 -- 172.21.15.138:0/845459743 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55806e57e780 con 0x55806e491c00 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55806d6b16c0 con 0x55806e491c00 2024-04-27T05:21:38.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4d90a6640 1 -- 172.21.15.138:0/845459743 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55806e492580 con 0x55806e491c00 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.769+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55806d6b11e0 con 0x55806e491c00 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4d90a6640 1 -- 172.21.15.138:0/845459743 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55806d6d6b00 con 0x55806e491c00 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4d90a6640 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e490800 0x55806d991700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4d90a6640 1 -- 172.21.15.138:0/845459743 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55806d6bfdc0 con 0x55806e491c00 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4f7c0f880 5 note: GC not initialized 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e490800 0x55806d991700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4d68a1640 20 reqs_thread_entry: start 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4f7c0f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4f48dd640 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e490800 0x55806d991700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55806e622a20 tx=0x55806d9f8960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4f7c0f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.770+0000 7fd4d489d640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4d407b640 20 reqs_thread_entry: start 2024-04-27T05:21:38.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db22000 con 0x55806da32400 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=385 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.771+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db22400 con 0x55806da32800 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:38.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db22800 con 0x55806da32800 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.772+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db22c00 con 0x55806da32800 2024-04-27T05:21:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.773+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 foo.buckets [call] v29'24 uv24 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.773+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 8.6 8:67982ffd:root::.bucket.meta.testx%3amyfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [create,call version.inc in=38b,writefull 0~371 in=371b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23000 con 0x55806da32400 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [create,call,writefull 0~371,setxattr (203)] v29'1 uv1 ondisk = 0) v8 ==== 338+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x7 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [notify cookie 94010082891264 in=818b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23400 con 0x55806e4a5c00 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== watch-notify(notify (1) cookie 94010082880896 notify 124554051595 ret 0) v3 ==== 848+0+0 (crc 0 0 0) 0x55806d9f7d40 con 0x55806e4a5c00 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4ee8d1640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051595 cookie 94010082880896 notifier 4589 bl.length()=806 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4ee8d1640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x7 2024-04-27T05:21:38.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4ee8d1640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4ee8d1640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 notify.1 [notify cookie 94010082891264 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4ee8d1640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42000 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 notify.1 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== watch-notify(notify_complete (2) cookie 94010082891264 notify 124554051595 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55806daab860 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 20 handle_bi_update(): bucket=testx:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_sources= new_sources= 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 20 handle_bi_update(): bucket=testx:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential sources added=[] removed=[] 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 20 handle_bi_update(): bucket=testx:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_dests= new_dests= 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 20 handle_bi_update(): bucket=testx:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential dests added=[] removed=[] 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.775+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23800 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.776+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v29'4 uv4 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.777+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23c00 con 0x55806e4a5c00 2024-04-27T05:21:38.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v29'1 uv1 ondisk = 0) v8 ==== 281+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f7c0f880 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f7c0f880 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f7c0f880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 94010082891264 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42000 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== watch-notify(notify (1) cookie 94010082884352 notify 124554051596 ret 0) v3 ==== 390+0+0 (crc 0 0 0) 0x55806daab860 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4ee8d1640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051596 cookie 94010082884352 notifier 4589 bl.length()=348 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4ee8d1640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4ee8d1640 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(71 notify.4 [notify cookie 94010082891264 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4ee8d1640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42400 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== watch-notify(notify_complete (2) cookie 94010082891264 notify 124554051596 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55806daaba00 con 0x55806e4a5c00 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.778+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42800 con 0x55806da32800 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.779+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(73 myfoo [call,delete] v29'25 uv25 ondisk = 0) v8 ==== 191+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.779+0000 7fd4f7c0f880 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.779+0000 7fd4f7c0f880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.779+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94010082891264 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42c00 con 0x55806da32400 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== watch-notify(notify (1) cookie 94010082886656 notify 124554051601 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x55806daaba00 con 0x55806da32400 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4ee0d0640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051601 cookie 94010082886656 notifier 4589 bl.length()=149 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(74 notify.6 [notify cookie 94010082891264 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4ee0d0640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db43800 con 0x55806da32400 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== watch-notify(notify_complete (2) cookie 94010082891264 notify 124554051601 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55806daaa000 con 0x55806da32400 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.780+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:76 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db43000 con 0x55806da32800 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(76 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call,delete] v29'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f7c0f880 10 removing default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 from cache 2024-04-27T05:21:38.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f7c0f880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [notify cookie 94010082891264 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db43400 con 0x55806da33400 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94010082882048 notify 124554051592 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x55806e5c6ea0 con 0x55806da33400 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4ee0d0640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051592 cookie 94010082882048 notifier 4589 bl.length()=206 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4ee0d0640 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806da33c00 con 0x55806da33400 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(77 notify.2 [notify cookie 94010082891264 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.781+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(78 notify.2 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.782+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94010082891264 notify 124554051592 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55806e5c7040 con 0x55806da33400 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.782+0000 7fd4db0aa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.782+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94010082888960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db22000 con 0x55806e4a5c00 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.783+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94010082888960] v29'135 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.783+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94010082879744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23000 con 0x55806da32800 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.784+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94010082879744] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.784+0000 7fd4f7c0f880 20 remove_watcher() i=0 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.784+0000 7fd4f7c0f880 2 removed watcher, disabling cache 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.784+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94010082880896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23400 con 0x55806e4a5c00 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.785+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94010082880896] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.785+0000 7fd4f7c0f880 20 remove_watcher() i=1 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.785+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94010082882048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23800 con 0x55806da33400 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.786+0000 7fd4f40dc640 1 -- 172.21.15.138:0/3897756181 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94010082882048] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46fc0 con 0x55806da33400 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.786+0000 7fd4f7c0f880 20 remove_watcher() i=2 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.786+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94010082883200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db23c00 con 0x55806da32800 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.787+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94010082883200] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da47200 con 0x55806da32800 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.787+0000 7fd4f7c0f880 20 remove_watcher() i=3 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.787+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94010082884352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490c00 con 0x55806e4a5c00 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.788+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94010082884352] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806e4a5c00 2024-04-27T05:21:38.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.788+0000 7fd4f7c0f880 20 remove_watcher() i=4 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.788+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94010082885504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806e490400 con 0x55806da32400 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.789+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94010082885504] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.789+0000 7fd4f7c0f880 20 remove_watcher() i=5 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.789+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94010082886656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42000 con 0x55806da32400 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.790+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94010082886656] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.790+0000 7fd4f7c0f880 20 remove_watcher() i=6 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.790+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94010082887808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55806db42400 con 0x55806da32400 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f48dd640 1 -- 172.21.15.138:0/3897756181 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94010082887808] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55806da46900 con 0x55806da32400 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 20 remove_watcher() i=7 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55806da32400 msgr2=0x55806d991180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55806da32400 0x55806d991180 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55806e4a5c00 msgr2=0x55806d98fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55806e4a5c00 0x55806d98fb80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55806da32800 msgr2=0x55806d990c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55806da32800 0x55806d990c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55806da33400 msgr2=0x55806d990100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55806da33400 0x55806d990100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806da32000 msgr2=0x55806d98f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806da32000 0x55806d98f600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55806d7b8c00 tx=0x55806d9f8120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 msgr2=0x55806d98eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3897756181 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e4a5800 0x55806d98eb00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55806e569950 tx=0x55806d7bf1a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 reap_dead start 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f38db640 1 -- 172.21.15.138:0/3897756181 reap_dead start 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 shutdown_connections 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 >> 172.21.15.138:0/3897756181 conn(0x55806d6ad400 msgr2=0x55806d7b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 shutdown_connections 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.791+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3897756181 wait complete. 2024-04-27T05:21:38.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e490800 msgr2=0x55806d991700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e490800 0x55806d991700 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55806e622a20 tx=0x55806d9f8960 comp rx=0 tx=0).stop 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 msgr2=0x55806d990680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55806da137a0 tx=0x55806d9f8000 comp rx=0 tx=0).stop 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 shutdown_connections 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e490800 0x55806d991700 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/845459743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806e491c00 0x55806d990680 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 >> 172.21.15.138:0/845459743 conn(0x55806e491400 msgr2=0x55806d7b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 shutdown_connections 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/845459743 wait complete. 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e602000 msgr2=0x55806d98f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e602000 0x55806d98f080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55806e5b2420 tx=0x55806d7bf4a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 msgr2=0x55806d98e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55806e569350 tx=0x55806d7bec60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 shutdown_connections 2024-04-27T05:21:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55806e602000 0x55806d98f080 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 --2- 172.21.15.138:0/3381747822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55806d6ad000 0x55806d98e580 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 >> 172.21.15.138:0/3381747822 conn(0x55806d6acc00 msgr2=0x55806d7b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 shutdown_connections 2024-04-27T05:21:38.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.792+0000 7fd4f7c0f880 1 -- 172.21.15.138:0/3381747822 wait complete. 2024-04-27T05:21:38.816 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-04-27T05:21:38.816 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-04-27T05:21:38.816 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-04-27T05:21:38.858 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:38.859 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef87f2c880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5561128b1400 0x556112b92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef87f2c880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556112b88600 con 0x5561128b1000 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef87f2c880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5561128f0280 con 0x5561128b1400 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef84105640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5561128b1400 0x556112b92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef84906640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5561128b1000 0x5561129b8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42852/0 (socket says 172.21.15.138:42852) 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.857+0000 7fef84906640 1 -- 172.21.15.138:0/3086826419 learned_addr learned my addr 172.21.15.138:0/3086826419 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:38.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.858+0000 7fef84105640 1 -- 172.21.15.138:0/3086826419 >> v1:172.21.15.114:6789/0 conn(0x5561128b1000 legacy=0x5561129b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.858+0000 7fef84105640 1 -- 172.21.15.138:0/3086826419 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561128b5380 con 0x5561128b1400 2024-04-27T05:21:38.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.858+0000 7fef83904640 1 -- 172.21.15.138:0/3086826419 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756889811 0 0) 0x5561128b49c0 con 0x5561128b1000 2024-04-27T05:21:38.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.859+0000 7fef84105640 1 --2- 172.21.15.138:0/3086826419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1400 0x556112b92580 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5561129bcc00 tx=0x5561129c2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601cc5428145418 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.859+0000 7fef83904640 1 -- 172.21.15.138:0/3086826419 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c9ce0 con 0x5561128b1400 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.859+0000 7fef83904640 1 -- 172.21.15.138:0/3086826419 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556112b89b00 con 0x5561128b1400 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.859+0000 7fef83904640 1 -- 172.21.15.138:0/3086826419 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c9e40 con 0x5561128b1400 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 -- 172.21.15.138:0/3086826419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1400 msgr2=0x556112b92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 --2- 172.21.15.138:0/3086826419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1400 0x556112b92580 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5561129bcc00 tx=0x5561129c2de0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 -- 172.21.15.138:0/3086826419 shutdown_connections 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 --2- 172.21.15.138:0/3086826419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1400 0x556112b92580 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 -- 172.21.15.138:0/3086826419 >> 172.21.15.138:0/3086826419 conn(0x5561128b0c00 msgr2=0x5561129b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 -- 172.21.15.138:0/3086826419 shutdown_connections 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.860+0000 7fef87f2c880 1 -- 172.21.15.138:0/3086826419 wait complete. 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef87f2c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef87f2c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561128f0780 con 0x5561128b1000 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef84906640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef84906640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59420/0 (socket says 172.21.15.138:59420) 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef84906640 1 -- 172.21.15.138:0/3911366094 learned_addr learned my addr 172.21.15.138:0/3911366094 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef84906640 1 -- 172.21.15.138:0/3911366094 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561128b4ea0 con 0x5561128b1000 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef84906640 1 --2- 172.21.15.138:0/3911366094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55611376cc90 tx=0x5561129c2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4604c3d6adc487d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.861+0000 7fef83103640 1 -- 172.21.15.138:0/3911366094 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c94a0 con 0x5561128b1000 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef83103640 1 -- 172.21.15.138:0/3911366094 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556112b89e00 con 0x5561128b1000 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef83103640 1 -- 172.21.15.138:0/3911366094 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c9ce0 con 0x5561128b1000 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- 172.21.15.138:0/3911366094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 msgr2=0x556112b92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 --2- 172.21.15.138:0/3911366094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55611376cc90 tx=0x5561129c2cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- 172.21.15.138:0/3911366094 shutdown_connections 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 --2- 172.21.15.138:0/3911366094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- 172.21.15.138:0/3911366094 >> 172.21.15.138:0/3911366094 conn(0x5561128b0c00 msgr2=0x5561129b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- 172.21.15.138:0/3911366094 shutdown_connections 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- 172.21.15.138:0/3911366094 wait complete. 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef87f2c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561128f08c0 con 0x5561128b1000 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef84906640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.862+0000 7fef84906640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59434/0 (socket says 172.21.15.138:59434) 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef84906640 1 -- 172.21.15.138:0/886444005 learned_addr learned my addr 172.21.15.138:0/886444005 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef84906640 1 -- 172.21.15.138:0/886444005 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561128b56c0 con 0x5561128b1000 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef84906640 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55611376d350 tx=0x5561129c2c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef81900640 1 -- 172.21.15.138:0/886444005 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c9e40 con 0x5561128b1000 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef81900640 1 -- 172.21.15.138:0/886444005 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556113782000 con 0x5561128b1000 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5561128b51e0 con 0x5561128b1000 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef81900640 1 -- 172.21.15.138:0/886444005 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c94a0 con 0x5561128b1000 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556112922ea0 con 0x5561128b1000 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.863+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef87f2c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef87f2c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561128f0c80 con 0x5561136a9800 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef81900640 1 -- 172.21.15.138:0/886444005 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5561128db600 con 0x5561128b1000 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef85107640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef85107640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59448/0 (socket says 172.21.15.138:59448) 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef85107640 1 -- 172.21.15.138:0/2008022762 learned_addr learned my addr 172.21.15.138:0/2008022762 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef81900640 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55611380e000 0x556112b93080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef81900640 1 -- 172.21.15.138:0/886444005 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5561128c3a40 con 0x5561128b1000 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef84906640 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55611380e000 0x556112b93080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef85107640 1 -- 172.21.15.138:0/2008022762 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556112922d00 con 0x5561136a9800 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef85107640 1 --2- 172.21.15.138:0/2008022762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5561137bd440 tx=0x5561129c3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691fe9d8b7e51628 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef7f0fb640 1 -- 172.21.15.138:0/2008022762 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c9e40 con 0x5561136a9800 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef7f0fb640 1 -- 172.21.15.138:0/2008022762 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556113782300 con 0x5561136a9800 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.864+0000 7fef7f0fb640 1 -- 172.21.15.138:0/2008022762 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c94a0 con 0x5561136a9800 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- 172.21.15.138:0/2008022762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 msgr2=0x556112b92b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef84906640 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55611380e000 0x556112b93080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5561137bc450 tx=0x5561129c34a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 --2- 172.21.15.138:0/2008022762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5561137bd440 tx=0x5561129c3a40 comp rx=0 tx=0).stop 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- 172.21.15.138:0/2008022762 shutdown_connections 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 --2- 172.21.15.138:0/2008022762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- 172.21.15.138:0/2008022762 >> 172.21.15.138:0/2008022762 conn(0x5561128b1400 msgr2=0x5561129bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- 172.21.15.138:0/2008022762 shutdown_connections 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- 172.21.15.138:0/2008022762 wait complete. 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef87f2c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561128d4280 con 0x5561136a9800 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef85107640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef85107640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59462/0 (socket says 172.21.15.138:59462) 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.865+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 learned_addr learned my addr 172.21.15.138:0/1245508763 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556112922b60 con 0x5561136a9800 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef85107640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55611376d9e0 tx=0x5561129c3aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef7d8f8640 1 -- 172.21.15.138:0/1245508763 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c9e40 con 0x5561136a9800 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef7d8f8640 1 -- 172.21.15.138:0/1245508763 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556113782480 con 0x5561136a9800 2024-04-27T05:21:38.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5561129229c0 con 0x5561136a9800 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef7d8f8640 1 -- 172.21.15.138:0/1245508763 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561128c94a0 con 0x5561136a9800 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.866+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556112922820 con 0x5561136a9800 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef7d8f8640 1 -- 172.21.15.138:0/1245508763 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5561128db8c0 con 0x5561136a9800 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef7d8f8640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556112c38000 0x556112b93600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef7d8f8640 1 -- 172.21.15.138:0/1245508763 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5561128c3c00 con 0x5561136a9800 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef84105640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556112c38000 0x556112b93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5561136a9c00 0x556112b93b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38400 con 0x5561136a9c00 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef85107640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5561136a9c00 0x556112b93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef84105640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556112c38000 0x556112b93600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x5561129bcc00 tx=0x556112c00120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.867+0000 7fef85107640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5561136a9c00 0x556112b93b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 realm 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38800 con 0x5561136a9c00 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38c00 con 0x5561136a9c00 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39000 con 0x5561136a9c00 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556112c39400 0x556112b94100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38400 con 0x556112c39400 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.868+0000 7fef84906640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556112c39400 0x556112b94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef84906640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556112c39400 0x556112b94100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef87f2c880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef87f2c880 20 searching for the correct realm 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38800 con 0x556112c39400 2024-04-27T05:21:38.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef84906640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556112c38800 0x556112b94c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.869+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38400 con 0x556112c38800 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef84105640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556112c38800 0x556112b94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef84105640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556112c38800 0x556112b94c00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef84105640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556112c38400 0x556112b95180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef85107640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556112c38400 0x556112b95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.870+0000 7fef85107640 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556112c38400 0x556112b95180 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.871+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38800 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.872+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38800 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.873+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.874+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38800 2024-04-27T05:21:38.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.875+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38800 2024-04-27T05:21:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.876+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c39400 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38400 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x5561136a9c00 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5561136a8800 con 0x556112c38800 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38c00 con 0x5561136a9c00 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.877+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39000 con 0x5561136a9c00 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39800 con 0x556112c39400 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 zone default found 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 4 Realm: () 2024-04-27T05:21:38.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 zonegroup default 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39c00 con 0x556112c38400 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.878+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690000 con 0x556112c39400 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690400 con 0x556112c39400 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 20 started sync module instance, tier type = 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.879+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690800 con 0x556112c38800 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.880+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.880+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93875414280448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690c00 con 0x556112c38800 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.880+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113691000 con 0x5561136a9c00 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.881+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93875414280448] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.881+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.881+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93875414281600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113691400 con 0x5561136a9c00 2024-04-27T05:21:38.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.881+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113691800 con 0x556112c39400 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.882+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93875414281600] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.882+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.882+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93875414282752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690000 con 0x556112c39400 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.882+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690400 con 0x556112c38800 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.883+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93875414282752] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.883+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.883+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93875414283904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113691800 con 0x556112c38800 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.883+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39800 con 0x5561136a9c00 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.884+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.884+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93875414285056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c38c00 con 0x5561136a9c00 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.884+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39000 con 0x556112c38400 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.884+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93875414283904] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.885+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93875414285056] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.885+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.885+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93875414286208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112c39c00 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.885+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55611380f800 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.886+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.886+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93875414287360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690800 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.886+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690c00 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.886+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93875414286208] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93875414287360] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93875414288512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113690400 con 0x556112c38400 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=0 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=1 2024-04-27T05:21:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=2 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=3 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=4 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=5 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.887+0000 7fef87f2c880 20 add_watcher() i=6 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.888+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93875414288512] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.888+0000 7fef87f2c880 20 add_watcher() i=7 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.888+0000 7fef87f2c880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.888+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113691800 con 0x5561136a9c00 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.888+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.888+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93875414289664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113691000 con 0x5561136a9c00 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.889+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93875414289664] v29'136 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.889+0000 7fef6b8d4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.889+0000 7fef87f2c880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.889+0000 7fef87f2c880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef87f2c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 0x556112b94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef87f2c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561128d48c0 con 0x556113691c00 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef84105640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 0x556112b94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef84105640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 0x556112b94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59466/0 (socket says 172.21.15.138:59466) 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef84105640 1 -- 172.21.15.138:0/1304164805 learned_addr learned my addr 172.21.15.138:0/1304164805 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef84105640 1 -- 172.21.15.138:0/1304164805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556112922680 con 0x556113691c00 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef84105640 1 --2- 172.21.15.138:0/1304164805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 0x556112b94680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x556112c1cc30 tx=0x556112c000c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b34e0a08c70518 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.890+0000 7fef6b0d3640 1 -- 172.21.15.138:0/1304164805 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556113692580 con 0x556113691c00 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef6b0d3640 1 -- 172.21.15.138:0/1304164805 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556113782000 con 0x556113691c00 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef6b0d3640 1 -- 172.21.15.138:0/1304164805 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5561136926e0 con 0x556113691c00 2024-04-27T05:21:38.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 -- 172.21.15.138:0/1304164805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 msgr2=0x556112b94680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1304164805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 0x556112b94680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x556112c1cc30 tx=0x556112c000c0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 -- 172.21.15.138:0/1304164805 shutdown_connections 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1304164805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556113691c00 0x556112b94680 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 -- 172.21.15.138:0/1304164805 >> 172.21.15.138:0/1304164805 conn(0x556113691400 msgr2=0x5561129bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 -- 172.21.15.138:0/1304164805 shutdown_connections 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 -- 172.21.15.138:0/1304164805 wait complete. 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.891+0000 7fef87f2c880 1 Processor -- start 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef87f2c880 1 -- start start 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef87f2c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 0x556112b95700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef87f2c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561128d4780 con 0x5561136a8800 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef84105640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 0x556112b95700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef84105640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 0x556112b95700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59480/0 (socket says 172.21.15.138:59480) 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef84105640 1 -- 172.21.15.138:0/2408176265 learned_addr learned my addr 172.21.15.138:0/2408176265 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef84105640 1 -- 172.21.15.138:0/2408176265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561128b4ea0 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef84105640 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 0x556112b95700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x556112c1d710 tx=0x556112c00000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.892+0000 7fef698d0640 1 -- 172.21.15.138:0/2408176265 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556113692840 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef698d0640 1 -- 172.21.15.138:0/2408176265 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556113782780 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5561128b56c0 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef698d0640 1 -- 172.21.15.138:0/2408176265 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556113692580 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561128b51e0 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef698d0640 1 -- 172.21.15.138:0/2408176265 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5561128dab00 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef698d0640 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556113690800 0x556112cf2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef698d0640 1 -- 172.21.15.138:0/2408176265 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5561128c3dc0 con 0x5561136a8800 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef87f2c880 5 note: GC not initialized 2024-04-27T05:21:38.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.893+0000 7fef85107640 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556113690800 0x556112cf2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef670cb640 20 reqs_thread_entry: start 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef87f2c880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef85107640 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556113690800 0x556112cf2000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55611382ea20 tx=0x556112c00960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef87f2c880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef650c7640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef648a5640 20 reqs_thread_entry: start 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef87f2c880 20 rados->read ofs=0 len=0 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.894+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1a400 con 0x5561136a9c00 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.895+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 281+0+332 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.895+0000 7fef87f2c880 20 rados_obj.operate() r=0 bl.length=193 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.895+0000 7fef6b8d4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.895+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93875414289664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1a800 con 0x5561136a9c00 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.896+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93875414289664] v29'137 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.896+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93875414280448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1ac00 con 0x556112c38800 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.897+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93875414280448] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.897+0000 7fef87f2c880 20 remove_watcher() i=0 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.897+0000 7fef87f2c880 2 removed watcher, disabling cache 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.897+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93875414281600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1b000 con 0x5561136a9c00 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.898+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93875414281600] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.898+0000 7fef87f2c880 20 remove_watcher() i=1 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.898+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93875414282752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1b400 con 0x556112c39400 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.899+0000 7fef84906640 1 -- 172.21.15.138:0/1245508763 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93875414282752] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cd80 con 0x556112c39400 2024-04-27T05:21:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.899+0000 7fef87f2c880 20 remove_watcher() i=2 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.899+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93875414283904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1b800 con 0x556112c38800 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.900+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93875414283904] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4cfc0 con 0x556112c38800 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.900+0000 7fef87f2c880 20 remove_watcher() i=3 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.900+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93875414285056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556112d1bc00 con 0x5561136a9c00 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.901+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93875414285056] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x5561136a9c00 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.901+0000 7fef87f2c880 20 remove_watcher() i=4 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.901+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93875414286208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113824000 con 0x556112c38400 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.902+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93875414286208] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.902+0000 7fef87f2c880 20 remove_watcher() i=5 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.902+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93875414287360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113824400 con 0x556112c38400 2024-04-27T05:21:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.903+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93875414287360] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.903+0000 7fef87f2c880 20 remove_watcher() i=6 2024-04-27T05:21:38.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.903+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93875414288512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556113824800 con 0x556112c38400 2024-04-27T05:21:38.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef85107640 1 -- 172.21.15.138:0/1245508763 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93875414288512] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556112c4c6c0 con 0x556112c38400 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 20 remove_watcher() i=7 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556112c38400 msgr2=0x556112b95180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556112c38400 0x556112b95180 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5561136a9c00 msgr2=0x556112b93b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5561136a9c00 0x556112b93b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556112c38800 msgr2=0x556112b94c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556112c38800 0x556112b94c00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556112c39400 msgr2=0x556112b94100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556112c39400 0x556112b94100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556112c38000 msgr2=0x556112b93600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556112c38000 0x556112b93600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x5561129bcc00 tx=0x556112c00120 comp rx=0 tx=0).stop 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef84105640 1 -- 172.21.15.138:0/1245508763 reap_dead start 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 msgr2=0x556112b92b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.904+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55611376d9e0 tx=0x5561129c3aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 shutdown_connections 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/1245508763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a9800 0x556112b92b00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 >> 172.21.15.138:0/1245508763 conn(0x5561128b1400 msgr2=0x5561129bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 shutdown_connections 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/1245508763 wait complete. 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556113690800 msgr2=0x556112cf2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556113690800 0x556112cf2000 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55611382ea20 tx=0x556112c00960 comp rx=0 tx=0).stop 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 msgr2=0x556112b95700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 0x556112b95700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x556112c1d710 tx=0x556112c00000 comp rx=0 tx=0).stop 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 shutdown_connections 2024-04-27T05:21:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556113690800 0x556112cf2000 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/2408176265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561136a8800 0x556112b95700 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 >> 172.21.15.138:0/2408176265 conn(0x556113691400 msgr2=0x5561129bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 shutdown_connections 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/2408176265 wait complete. 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55611380e000 msgr2=0x556112b93080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55611380e000 0x556112b93080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5561137bc450 tx=0x5561129c34a0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 msgr2=0x556112b92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.905+0000 7fef87f2c880 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55611376d350 tx=0x5561129c2c60 comp rx=0 tx=0).stop 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.906+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 shutdown_connections 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.906+0000 7fef87f2c880 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55611380e000 0x556112b93080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.906+0000 7fef87f2c880 1 --2- 172.21.15.138:0/886444005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5561128b1000 0x556112b92580 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.906+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 >> 172.21.15.138:0/886444005 conn(0x5561128b0c00 msgr2=0x5561129b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.906+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 shutdown_connections 2024-04-27T05:21:38.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:38.906+0000 7fef87f2c880 1 -- 172.21.15.138:0/886444005 wait complete. 2024-04-27T05:21:38.926 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:38.926 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "bucket:testx/myfoo", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": { 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tag": "oQ+60edTHlaK89d1HB9KSKUC", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": 2 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "mtime": "2024-04-27T05:21:38.778655Z", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "data": { 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": { 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "myfoo", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "marker": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_id": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tenant": "testx", 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "explicit_placement": { 2024-04-27T05:21:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "", 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "", 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "" 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "testx$tenanteduser", 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: "creation_time": "2024-04-27T05:21:36.328858Z", 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: "linked": true, 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: "has_bucket_info": false 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:38.928 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:38.928 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'oQ+60edTHlaK89d1HB9KSKUC', 'ver': 2}, 'mtime': '2024-04-27T05:21:38.778655Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'bucket_id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-04-27T05:21:36.328858Z', 'linked': True, 'has_bucket_info': False}} 2024-04-27T05:21:38.929 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-27T05:21:38.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-27T05:21:38.929 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-04-27T05:21:39.011 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.011 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.009+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.009+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.009+0000 7efea7726880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.009+0000 7efea7726880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55963ba26600 con 0x55963b74f400 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.009+0000 7efea7726880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55963b78e280 con 0x55963b74f000 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.010+0000 7efea3fd4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.010+0000 7efea37d3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55963b74f400 0x55963b856800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42858/0 (socket says 172.21.15.138:42858) 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.010+0000 7efea37d3640 1 -- 172.21.15.138:0/4136331526 learned_addr learned my addr 172.21.15.138:0/4136331526 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.010+0000 7efea3fd4640 1 -- 172.21.15.138:0/4136331526 >> v1:172.21.15.114:6789/0 conn(0x55963b74f400 legacy=0x55963b856800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.010+0000 7efea3fd4640 1 -- 172.21.15.138:0/4136331526 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b753380 con 0x55963b74f000 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea3fd4640 1 --2- 172.21.15.138:0/4136331526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55963b85ad50 tx=0x55963b860de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c935388a656a629 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea2fd2640 1 -- 172.21.15.138:0/4136331526 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b767ce0 con 0x55963b74f000 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea2fd2640 1 -- 172.21.15.138:0/4136331526 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963ba27b00 con 0x55963b74f000 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea2fd2640 1 -- 172.21.15.138:0/4136331526 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b767e40 con 0x55963b74f000 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 -- 172.21.15.138:0/4136331526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 msgr2=0x55963ba30580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 --2- 172.21.15.138:0/4136331526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55963b85ad50 tx=0x55963b860de0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 -- 172.21.15.138:0/4136331526 shutdown_connections 2024-04-27T05:21:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 --2- 172.21.15.138:0/4136331526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 -- 172.21.15.138:0/4136331526 >> 172.21.15.138:0/4136331526 conn(0x55963b74ec00 msgr2=0x55963b856000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 -- 172.21.15.138:0/4136331526 shutdown_connections 2024-04-27T05:21:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.012+0000 7efea7726880 1 -- 172.21.15.138:0/4136331526 wait complete. 2024-04-27T05:21:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea7726880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea7726880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b78e780 con 0x55963b74f000 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea3fd4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea3fd4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59496/0 (socket says 172.21.15.138:59496) 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.013+0000 7efea3fd4640 1 -- 172.21.15.138:0/2928868945 learned_addr learned my addr 172.21.15.138:0/2928868945 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea3fd4640 1 -- 172.21.15.138:0/2928868945 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b752ea0 con 0x55963b74f000 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea3fd4640 1 --2- 172.21.15.138:0/2928868945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55963c553560 tx=0x55963b860cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c4d1fbe25bc08c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea27d1640 1 -- 172.21.15.138:0/2928868945 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b7674a0 con 0x55963b74f000 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea27d1640 1 -- 172.21.15.138:0/2928868945 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963ba27e00 con 0x55963b74f000 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea27d1640 1 -- 172.21.15.138:0/2928868945 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b767ce0 con 0x55963b74f000 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 -- 172.21.15.138:0/2928868945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 msgr2=0x55963ba30580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 --2- 172.21.15.138:0/2928868945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55963c553560 tx=0x55963b860cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 -- 172.21.15.138:0/2928868945 shutdown_connections 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 --2- 172.21.15.138:0/2928868945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 -- 172.21.15.138:0/2928868945 >> 172.21.15.138:0/2928868945 conn(0x55963b74ec00 msgr2=0x55963b857000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 -- 172.21.15.138:0/2928868945 shutdown_connections 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.014+0000 7efea7726880 1 -- 172.21.15.138:0/2928868945 wait complete. 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea7726880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea7726880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b78e8c0 con 0x55963b74f000 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea3fd4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea3fd4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59504/0 (socket says 172.21.15.138:59504) 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea3fd4640 1 -- 172.21.15.138:0/3952602427 learned_addr learned my addr 172.21.15.138:0/3952602427 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea3fd4640 1 -- 172.21.15.138:0/3952602427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b7536c0 con 0x55963b74f000 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.015+0000 7efea3fd4640 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55963c60ade0 tx=0x55963b860c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea0fce640 1 -- 172.21.15.138:0/3952602427 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b767e40 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea0fce640 1 -- 172.21.15.138:0/3952602427 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963c52e000 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55963b7531e0 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea0fce640 1 -- 172.21.15.138:0/3952602427 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b7674a0 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55963b7c0ea0 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea0fce640 1 -- 172.21.15.138:0/3952602427 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55963b778b00 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea7726880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.016+0000 7efea7726880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b772280 con 0x55963c546c00 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea0fce640 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c547c00 0x55963ba31080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea0fce640 1 -- 172.21.15.138:0/3952602427 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55963b761a40 con 0x55963b74f000 2024-04-27T05:21:39.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea47d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea47d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59518/0 (socket says 172.21.15.138:59518) 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea47d5640 1 -- 172.21.15.138:0/3200515810 learned_addr learned my addr 172.21.15.138:0/3200515810 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea3fd4640 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c547c00 0x55963ba31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea47d5640 1 -- 172.21.15.138:0/3200515810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b7c0d00 con 0x55963c546c00 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea3fd4640 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c547c00 0x55963ba31080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55963c64a2a0 tx=0x55963b860cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efea47d5640 1 --2- 172.21.15.138:0/3200515810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55963c60b590 tx=0x55963b861c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bf0a61d58b8c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efe9e7c9640 1 -- 172.21.15.138:0/3200515810 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b767e40 con 0x55963c546c00 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efe9e7c9640 1 -- 172.21.15.138:0/3200515810 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963c52e600 con 0x55963c546c00 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.017+0000 7efe9e7c9640 1 -- 172.21.15.138:0/3200515810 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b7674a0 con 0x55963c546c00 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- 172.21.15.138:0/3200515810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 msgr2=0x55963ba30b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 --2- 172.21.15.138:0/3200515810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55963c60b590 tx=0x55963b861c20 comp rx=0 tx=0).stop 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- 172.21.15.138:0/3200515810 shutdown_connections 2024-04-27T05:21:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 --2- 172.21.15.138:0/3200515810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- 172.21.15.138:0/3200515810 >> 172.21.15.138:0/3200515810 conn(0x55963b74f400 msgr2=0x55963b859000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- 172.21.15.138:0/3200515810 shutdown_connections 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- 172.21.15.138:0/3200515810 wait complete. 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea7726880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b773180 con 0x55963c546c00 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea47d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea47d5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59526/0 (socket says 172.21.15.138:59526) 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 learned_addr learned my addr 172.21.15.138:0/3828805424 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.018+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b7c0b60 con 0x55963c546c00 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efea47d5640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55963c5535c0 tx=0x55963b861980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efe9cfc6640 1 -- 172.21.15.138:0/3828805424 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b767e40 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efe9cfc6640 1 -- 172.21.15.138:0/3828805424 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963c52e780 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55963b7c09c0 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efe9cfc6640 1 -- 172.21.15.138:0/3828805424 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963b7674a0 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55963b7c0820 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efe9cfc6640 1 -- 172.21.15.138:0/3828805424 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55963b779600 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efe9cfc6640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963bacc000 0x55963ba31600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.019+0000 7efe9cfc6640 1 -- 172.21.15.138:0/3828805424 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55963b761c00 con 0x55963c546c00 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea37d3640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963bacc000 0x55963ba31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55963c547000 0x55963ba31b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c547400 con 0x55963c547000 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea47d5640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55963c547000 0x55963ba31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea37d3640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963bacc000 0x55963ba31600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55963c6ad260 tx=0x55963ba90120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea47d5640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55963c547000 0x55963ba31b80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 20 realm 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.020+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c547800 con 0x55963c547000 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacd000 con 0x55963c547000 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacd400 con 0x55963c547000 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55963bacd800 0x55963ba32100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacdc00 con 0x55963bacd800 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea3fd4640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55963bacd800 0x55963ba32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.021+0000 7efea3fd4640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55963bacd800 0x55963ba32100 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea7726880 20 searching for the correct realm 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacd000 con 0x55963bacd800 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea3fd4640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55963bacd000 0x55963ba32c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacdc00 con 0x55963bacd000 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.022+0000 7efea37d3640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55963bacd000 0x55963ba32c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.023+0000 7efea37d3640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55963bacd000 0x55963ba32c00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.023+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.023+0000 7efea37d3640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55963bacdc00 0x55963ba33180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.023+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.023+0000 7efea47d5640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55963bacdc00 0x55963ba33180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.023+0000 7efea47d5640 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55963bacdc00 0x55963ba33180 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.024+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd000 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.025+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd000 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.026+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.027+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.028+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd000 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd800 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.029+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacdc00 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963c547000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baccc00 con 0x55963bacd000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacd400 con 0x55963c547000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c547400 con 0x55963c547000 2024-04-27T05:21:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.030+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c547800 con 0x55963bacd800 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 zone default found 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 4 Realm: () 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 zonegroup default 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0000 con 0x55963bacdc00 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0400 con 0x55963bacd800 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.031+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0800 con 0x55963bacd800 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.032+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.032+0000 7efea7726880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.032+0000 7efea7726880 20 started sync module instance, tier type = 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.032+0000 7efea7726880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.032+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0c00 con 0x55963bacd000 2024-04-27T05:21:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.033+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.033+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94103733963008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1000 con 0x55963bacd000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.033+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1400 con 0x55963c547000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.033+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94103733963008] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.034+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.034+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94103733964160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1800 con 0x55963c547000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.034+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1c00 con 0x55963bacd800 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.035+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.035+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94103733965312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0400 con 0x55963bacd800 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.035+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0800 con 0x55963bacd000 2024-04-27T05:21:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.035+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94103733964160] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.036+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.036+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94103733966464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1c00 con 0x55963bacd000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.036+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94103733965312] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.036+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0000 con 0x55963c547000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.036+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.037+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94103733967616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c547800 con 0x55963c547000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.037+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c547400 con 0x55963bacdc00 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.037+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94103733966464] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.037+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.037+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94103733968768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bacd400 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.038+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c6aec00 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.038+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94103733967616] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.038+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94103733968768] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.039+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.039+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94103733969920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963c6ae800 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.039+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0c00 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.039+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94103733969920] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94103733971072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1000 con 0x55963bacdc00 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=0 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=1 2024-04-27T05:21:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=2 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=3 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=4 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=5 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=6 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94103733971072] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 20 add_watcher() i=7 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.040+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0800 con 0x55963c547000 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.041+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.041+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94103733972224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf1400 con 0x55963c547000 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94103733972224] v29'138 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efe8afa2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea7726880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea7726880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea7726880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea7726880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b772780 con 0x55963baccc00 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea37d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea37d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59536/0 (socket says 172.21.15.138:59536) 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.042+0000 7efea37d3640 1 -- 172.21.15.138:0/1528905855 learned_addr learned my addr 172.21.15.138:0/1528905855 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.043+0000 7efea37d3640 1 -- 172.21.15.138:0/1528905855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b7c0680 con 0x55963baccc00 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.043+0000 7efea37d3640 1 --2- 172.21.15.138:0/1528905855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55963baae930 tx=0x55963ba90180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988a57a1e2d7a2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.043+0000 7efe8a7a1640 1 -- 172.21.15.138:0/1528905855 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963baf2580 con 0x55963baccc00 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.043+0000 7efe8a7a1640 1 -- 172.21.15.138:0/1528905855 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963c52e000 con 0x55963baccc00 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.043+0000 7efe8a7a1640 1 -- 172.21.15.138:0/1528905855 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963baf26e0 con 0x55963baccc00 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- 172.21.15.138:0/1528905855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 msgr2=0x55963ba32680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 --2- 172.21.15.138:0/1528905855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55963baae930 tx=0x55963ba90180 comp rx=0 tx=0).stop 2024-04-27T05:21:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- 172.21.15.138:0/1528905855 shutdown_connections 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 --2- 172.21.15.138:0/1528905855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- 172.21.15.138:0/1528905855 >> 172.21.15.138:0/1528905855 conn(0x55963baf1800 msgr2=0x55963b859800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- 172.21.15.138:0/1528905855 shutdown_connections 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- 172.21.15.138:0/1528905855 wait complete. 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 Processor -- start 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- start start 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea7726880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55963b772500 con 0x55963baccc00 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea37d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea37d3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59540/0 (socket says 172.21.15.138:59540) 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.044+0000 7efea37d3640 1 -- 172.21.15.138:0/3949368295 learned_addr learned my addr 172.21.15.138:0/3949368295 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efea37d3640 1 -- 172.21.15.138:0/3949368295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55963b7c04e0 con 0x55963baccc00 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efea37d3640 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55963baaf890 tx=0x55963ba90ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efe88f9e640 1 -- 172.21.15.138:0/3949368295 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963baf2840 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efe88f9e640 1 -- 172.21.15.138:0/3949368295 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55963c52ea80 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55963b753380 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efe88f9e640 1 -- 172.21.15.138:0/3949368295 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55963baf2580 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.045+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55963b752ea0 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efe88f9e640 1 -- 172.21.15.138:0/3949368295 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55963b779b80 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efe88f9e640 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c6aec00 0x55963ba33700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efe88f9e640 1 -- 172.21.15.138:0/3949368295 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55963b761dc0 con 0x55963baccc00 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea7726880 5 note: GC not initialized 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea47d5640 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c6aec00 0x55963ba33700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efe86799640 20 reqs_thread_entry: start 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea7726880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea47d5640 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c6aec00 0x55963ba33700 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55963c6ac6c0 tx=0x55963ba908a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea7726880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efe84795640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efe83f73640 20 reqs_thread_entry: start 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea7726880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.046+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc2000 con 0x55963c547000 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc2400 con 0x55963c547000 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+241 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=193 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.047+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.6 8:67982ffd:root::.bucket.meta.testx%3amyfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc2800 con 0x55963bacdc00 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~371 out=371b,getxattrs out=298b] v0'0 uv1 ondisk = 0) v8 ==== 338+0+733 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 20 rados_obj.operate() r=0 bl.length=371 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.048+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc2c00 con 0x55963c547000 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.049+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v29'5 uv5 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.049+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [create,call version.inc in=38b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc3000 con 0x55963bacd000 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [create,call,writefull 0~352,setxattr (123)] v29'7 uv7 ondisk = 0) v8 ==== 332+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea7726880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x7 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea7726880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea7726880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea7726880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94103733974528 in=713b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc3400 con 0x55963bacd800 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94103733965312 notify 124554051593 ret 0) v3 ==== 743+0+0 (crc 0 0 0) 0x55963c650d00 con 0x55963bacd800 2024-04-27T05:21:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efe9e7c9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051593 cookie 94103733965312 notifier 4605 bl.length()=701 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efe9e7c9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x7 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efe9e7c9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efe9e7c9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(67 notify.2 [notify cookie 94103733974528 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.050+0000 7efe9e7c9640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafc000 con 0x55963bacd800 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94103733974528 notify 124554051593 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55963c650ea0 con 0x55963bacd800 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea7726880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_sources= new_sources= 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea7726880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential sources added=[] removed=[] 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea7726880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_dests= new_dests= 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea7726880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential dests added=[] removed=[] 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.051+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc3800 con 0x55963bacd000 2024-04-27T05:21:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.052+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(69 foo.buckets [call] v29'26 uv26 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.052+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc3c00 con 0x55963bacd000 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v29'27 uv27 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea7726880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea7726880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea7726880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94103733974528 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafc800 con 0x55963bacdc00 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== watch-notify(notify (1) cookie 94103733969920 notify 124554051602 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55963ba8fd40 con 0x55963bacdc00 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efe9dfc8640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051602 cookie 94103733969920 notifier 4605 bl.length()=314 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [notify cookie 94103733974528 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efe9dfc8640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T05:21:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efe9dfc8640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.053+0000 7efe9dfc8640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963baf0c00 con 0x55963bacdc00 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.054+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.054+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== watch-notify(notify_complete (2) cookie 94103733974528 notify 124554051602 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55963bb4ba00 con 0x55963bacdc00 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.054+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafcc00 con 0x55963c547000 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(73 testx/myfoo [call,delete] v29'2 uv2 ondisk = 0) v8 ==== 197+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea7726880 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea7726880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94103733974528 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafd000 con 0x55963c547000 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== watch-notify(notify (1) cookie 94103733967616 notify 124554051597 ret 0) v3 ==== 197+0+0 (crc 0 0 0) 0x55963bb4ba00 con 0x55963c547000 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efe9dfc8640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051597 cookie 94103733967616 notifier 4605 bl.length()=155 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efe9dfc8640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc2000 con 0x55963c547000 2024-04-27T05:21:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(74 notify.4 [notify cookie 94103733974528 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== watch-notify(notify_complete (2) cookie 94103733974528 notify 124554051597 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55963bb4bba0 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.055+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 8.6 8:67982ffd:root::.bucket.meta.testx%3amyfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafd400 con 0x55963bacdc00 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call,delete] v29'2 uv2 ondisk = 0) v8 ==== 254+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea7726880 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 from cache 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea7726880 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 7.0 7:05bf5b68:::notify.1:head [notify cookie 94103733974528 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafd800 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== watch-notify(notify (1) cookie 94103733964160 notify 124554051598 ret 0) v3 ==== 254+0+0 (crc 0 0 0) 0x55963bb4bba0 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efe9e7c9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051598 cookie 94103733964160 notifier 4605 bl.length()=212 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efe9e7c9640 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafc400 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(77 notify.1 [notify cookie 94103733974528 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(78 notify.1 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.057+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== watch-notify(notify_complete (2) cookie 94103733974528 notify 124554051598 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55963bb4a1a0 con 0x55963c547000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.058+0000 7efe8afa2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.058+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94103733972224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafdc00 con 0x55963c547000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.059+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94103733972224] v29'139 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.059+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94103733963008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafcc00 con 0x55963bacd000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.061+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94103733963008] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.061+0000 7efea7726880 20 remove_watcher() i=0 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.061+0000 7efea7726880 2 removed watcher, disabling cache 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.061+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94103733964160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafd000 con 0x55963c547000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.062+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94103733964160] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.062+0000 7efea7726880 20 remove_watcher() i=1 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.062+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94103733965312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafd400 con 0x55963bacd800 2024-04-27T05:21:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.064+0000 7efea3fd4640 1 -- 172.21.15.138:0/3828805424 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94103733965312] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6d80 con 0x55963bacd800 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.064+0000 7efea7726880 20 remove_watcher() i=2 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.064+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94103733966464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafd800 con 0x55963bacd000 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.064+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94103733966464] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae6fc0 con 0x55963bacd000 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.064+0000 7efea7726880 20 remove_watcher() i=3 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.064+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94103733967616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafc400 con 0x55963c547000 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.066+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94103733967616] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963c547000 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.066+0000 7efea7726880 20 remove_watcher() i=4 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.066+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94103733968768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafdc00 con 0x55963bacdc00 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.067+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94103733968768] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.067+0000 7efea7726880 20 remove_watcher() i=5 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.067+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94103733969920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bafc800 con 0x55963bacdc00 2024-04-27T05:21:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.067+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94103733969920] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.067+0000 7efea7726880 20 remove_watcher() i=6 2024-04-27T05:21:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.068+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94103733971072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55963bbc2000 con 0x55963bacdc00 2024-04-27T05:21:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.068+0000 7efea47d5640 1 -- 172.21.15.138:0/3828805424 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94103733971072] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55963bae66c0 con 0x55963bacdc00 2024-04-27T05:21:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.068+0000 7efea7726880 20 remove_watcher() i=7 2024-04-27T05:21:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55963bacdc00 msgr2=0x55963ba33180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55963bacdc00 0x55963ba33180 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55963c547000 msgr2=0x55963ba31b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55963c547000 0x55963ba31b80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55963bacd000 msgr2=0x55963ba32c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55963bacd000 0x55963ba32c00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55963bacd800 msgr2=0x55963ba32100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55963bacd800 0x55963ba32100 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963bacc000 msgr2=0x55963ba31600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963bacc000 0x55963ba31600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55963c6ad260 tx=0x55963ba90120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 msgr2=0x55963ba30b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3828805424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963c546c00 0x55963ba30b00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55963c5535c0 tx=0x55963b861980 comp rx=0 tx=0).stop 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 reap_dead start 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea37d3640 1 -- 172.21.15.138:0/3828805424 reap_dead start 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 shutdown_connections 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 >> 172.21.15.138:0/3828805424 conn(0x55963b74f400 msgr2=0x55963b859000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 shutdown_connections 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3828805424 wait complete. 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c6aec00 msgr2=0x55963ba33700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c6aec00 0x55963ba33700 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55963c6ac6c0 tx=0x55963ba908a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 msgr2=0x55963ba32680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x55963baaf890 tx=0x55963ba90ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 shutdown_connections 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c6aec00 0x55963ba33700 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 --2- 172.21.15.138:0/3949368295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963baccc00 0x55963ba32680 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 >> 172.21.15.138:0/3949368295 conn(0x55963baf1800 msgr2=0x55963b859800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 shutdown_connections 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.069+0000 7efea7726880 1 -- 172.21.15.138:0/3949368295 wait complete. 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c547c00 msgr2=0x55963ba31080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c547c00 0x55963ba31080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55963c64a2a0 tx=0x55963b860cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 msgr2=0x55963ba30580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55963c60ade0 tx=0x55963b860c60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 shutdown_connections 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55963c547c00 0x55963ba31080 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 --2- 172.21.15.138:0/3952602427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55963b74f000 0x55963ba30580 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 >> 172.21.15.138:0/3952602427 conn(0x55963b74ec00 msgr2=0x55963b857000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 shutdown_connections 2024-04-27T05:21:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.070+0000 7efea7726880 1 -- 172.21.15.138:0/3952602427 wait complete. 2024-04-27T05:21:39.091 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T05:21:39.092 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T05:21:39.092 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-04-27T05:21:39.136 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.136 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.134+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.134+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.134+0000 7f5ec0817880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x560023e55400 0x560024136580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.134+0000 7f5ec0817880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56002412c600 con 0x560023e55000 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.134+0000 7f5ec0817880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x560023e94280 con 0x560023e55400 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.135+0000 7f5ebc4fa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x560023e55400 0x560024136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.135+0000 7f5ebccfb640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x560023e55000 0x560023f5c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42866/0 (socket says 172.21.15.138:42866) 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.135+0000 7f5ebccfb640 1 -- 172.21.15.138:0/3081129982 learned_addr learned my addr 172.21.15.138:0/3081129982 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.135+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/3081129982 >> v2:172.21.15.114:3300/0 conn(0x560023e55400 0x560024136580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59552/0 (socket says 172.21.15.138:59552) 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.135+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/3081129982 >> v1:172.21.15.114:6789/0 conn(0x560023e55000 legacy=0x560023f5c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.135+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/3081129982 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023e59380 con 0x560023e55400 2024-04-27T05:21:39.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.136+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/3081129982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136580 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x560023f60c00 tx=0x560023f66de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a7e494f681b18e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ebbcf9640 1 -- 172.21.15.138:0/3081129982 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6dce0 con 0x560023e55400 2024-04-27T05:21:39.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ebbcf9640 1 -- 172.21.15.138:0/3081129982 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56002412db00 con 0x560023e55400 2024-04-27T05:21:39.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ebbcf9640 1 -- 172.21.15.138:0/3081129982 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6de40 con 0x560023e55400 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 -- 172.21.15.138:0/3081129982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 msgr2=0x560024136580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3081129982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136580 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x560023f60c00 tx=0x560023f66de0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 -- 172.21.15.138:0/3081129982 shutdown_connections 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3081129982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136580 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 -- 172.21.15.138:0/3081129982 >> 172.21.15.138:0/3081129982 conn(0x560023e54c00 msgr2=0x560023f5c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 -- 172.21.15.138:0/3081129982 shutdown_connections 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.137+0000 7f5ec0817880 1 -- 172.21.15.138:0/3081129982 wait complete. 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ec0817880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 0x560024136580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ec0817880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560023e94780 con 0x560023e55000 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ebccfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 0x560024136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ebccfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 0x560024136580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59556/0 (socket says 172.21.15.138:59556) 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ebccfb640 1 -- 172.21.15.138:0/3140111553 learned_addr learned my addr 172.21.15.138:0/3140111553 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ebccfb640 1 -- 172.21.15.138:0/3140111553 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023e58ea0 con 0x560023e55000 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.138+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/3140111553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 0x560024136580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x560024d10cc0 tx=0x560023f66cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f3b47ac1cd8730 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ebb4f8640 1 -- 172.21.15.138:0/3140111553 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6d4a0 con 0x560023e55000 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ebb4f8640 1 -- 172.21.15.138:0/3140111553 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56002412de00 con 0x560023e55000 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ebb4f8640 1 -- 172.21.15.138:0/3140111553 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6dce0 con 0x560023e55000 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- 172.21.15.138:0/3140111553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 msgr2=0x560024136580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3140111553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 0x560024136580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x560024d10cc0 tx=0x560023f66cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- 172.21.15.138:0/3140111553 shutdown_connections 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3140111553 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55000 0x560024136580 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- 172.21.15.138:0/3140111553 >> 172.21.15.138:0/3140111553 conn(0x560023e54c00 msgr2=0x560023f5d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- 172.21.15.138:0/3140111553 shutdown_connections 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- 172.21.15.138:0/3140111553 wait complete. 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.139+0000 7f5ec0817880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560023e948c0 con 0x560023e55400 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ebccfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ebccfb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59566/0 (socket says 172.21.15.138:59566) 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ebccfb640 1 -- 172.21.15.138:0/3000908217 learned_addr learned my addr 172.21.15.138:0/3000908217 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ebccfb640 1 -- 172.21.15.138:0/3000908217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023e596c0 con 0x560023e55400 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136b00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x560024d112f0 tx=0x560023f66c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5eb9cf5640 1 -- 172.21.15.138:0/3000908217 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6de40 con 0x560023e55400 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560023e591e0 con 0x560023e55400 2024-04-27T05:21:39.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5eb9cf5640 1 -- 172.21.15.138:0/3000908217 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560024d28000 con 0x560023e55400 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5eb9cf5640 1 -- 172.21.15.138:0/3000908217 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6d4a0 con 0x560023e55400 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.140+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560023ec6ea0 con 0x560023e55400 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5eb9cf5640 1 -- 172.21.15.138:0/3000908217 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560023e7f600 con 0x560023e55400 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ec0817880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ec0817880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560023e78280 con 0x560024c4dc00 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5eb9cf5640 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024da6000 0x560024138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5eb9cf5640 1 -- 172.21.15.138:0/3000908217 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x560023e67a40 con 0x560023e55400 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ebd4fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024da6000 0x560024138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ebd4fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59578/0 (socket says 172.21.15.138:59578) 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/4068988523 learned_addr learned my addr 172.21.15.138:0/4068988523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/4068988523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023ec6d00 con 0x560024c4dc00 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.141+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024da6000 0x560024138100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x560024d564e0 tx=0x560023f674a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/4068988523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x560024d11410 tx=0x560023f67c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1153731ab608b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5eb74f0640 1 -- 172.21.15.138:0/4068988523 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6de40 con 0x560024c4dc00 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5eb74f0640 1 -- 172.21.15.138:0/4068988523 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560024d28300 con 0x560024c4dc00 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5eb74f0640 1 -- 172.21.15.138:0/4068988523 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6d4a0 con 0x560024c4dc00 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 -- 172.21.15.138:0/4068988523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 msgr2=0x560024137080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 --2- 172.21.15.138:0/4068988523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x560024d11410 tx=0x560023f67c20 comp rx=0 tx=0).stop 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 -- 172.21.15.138:0/4068988523 shutdown_connections 2024-04-27T05:21:39.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 --2- 172.21.15.138:0/4068988523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 -- 172.21.15.138:0/4068988523 >> 172.21.15.138:0/4068988523 conn(0x560024c4d800 msgr2=0x560023f5f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 -- 172.21.15.138:0/4068988523 shutdown_connections 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 -- 172.21.15.138:0/4068988523 wait complete. 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.142+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ec0817880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ec0817880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560023e79180 con 0x560024c4dc00 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ebd4fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ebd4fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59586/0 (socket says 172.21.15.138:59586) 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 learned_addr learned my addr 172.21.15.138:0/1947523028 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023ec6b60 con 0x560024c4dc00 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x560024d11980 tx=0x560023f67a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5eb5ced640 1 -- 172.21.15.138:0/1947523028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6de40 con 0x560024c4dc00 2024-04-27T05:21:39.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5eb5ced640 1 -- 172.21.15.138:0/1947523028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560024d28480 con 0x560024c4dc00 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560023ec69c0 con 0x560024c4dc00 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5eb5ced640 1 -- 172.21.15.138:0/1947523028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560023e6d4a0 con 0x560024c4dc00 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.143+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560023ec6820 con 0x560024c4dc00 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5eb5ced640 1 -- 172.21.15.138:0/1947523028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560023e7f8c0 con 0x560024c4dc00 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5eb5ced640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024dd3000 0x560024138680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5eb5ced640 1 -- 172.21.15.138:0/1947523028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x560023e67c00 con 0x560024c4dc00 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024dd3000 0x560024138680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd3400 0x560024137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd3800 con 0x560024dd3400 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd3400 0x560024137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.144+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024dd3000 0x560024138680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560023f60c00 tx=0x5600241aa120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd3400 0x560024137600 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 realm 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd3c00 con 0x560024dd3400 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4000 con 0x560024dd3400 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4400 con 0x560024dd3400 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.145+0000 7f5ec0817880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4800 0x560024137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4c00 con 0x5600241f4800 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4800 0x560024137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4800 0x560024137b80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ec0817880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ec0817880 20 searching for the correct realm 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.146+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5000 con 0x5600241f4800 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5600241f5000 0x560024136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4c00 con 0x5600241f5000 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5600241f5000 0x560024136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5600241f5000 0x560024136580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4c00 0x560024138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.147+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4c00 0x560024138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4c00 0x560024138c00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.148+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f5000 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.149+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f5000 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.150+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.151+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f5000 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.152+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f5000 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.153+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4800 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f4c00 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x5600241f5000 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ec0817880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ec0817880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ec0817880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ec0817880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5400 con 0x560024dd3400 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.154+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4000 con 0x560024dd3400 2024-04-27T05:21:39.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5800 con 0x5600241f4800 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 zone default found 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 4 Realm: () 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 zonegroup default 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5c00 con 0x5600241f4c00 2024-04-27T05:21:39.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd2800 con 0x5600241f4800 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.155+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.156+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd2400 con 0x5600241f4800 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.156+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.156+0000 7f5ec0817880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.156+0000 7f5ec0817880 20 started sync module instance, tier type = 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.156+0000 7f5ec0817880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.156+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd3800 con 0x5600241f5000 2024-04-27T05:21:39.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.157+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.157+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94558605207808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd3c00 con 0x5600241f5000 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.157+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd2800 con 0x560024dd3400 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.158+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94558605207808] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.158+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.158+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94558605208960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd2400 con 0x560024dd3400 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.158+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5800 con 0x5600241f4800 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.159+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94558605208960] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.159+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.159+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94558605210112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7400 con 0x5600241f4800 2024-04-27T05:21:39.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.159+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7000 con 0x5600241f5000 2024-04-27T05:21:39.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.160+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.160+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94558605211264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da6c00 con 0x5600241f5000 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.160+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da6800 con 0x560024dd3400 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.161+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94558605210112] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x5600241f4800 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.161+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94558605211264] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.161+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.162+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94558605212416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd3800 con 0x560024dd3400 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.162+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd3c00 con 0x5600241f4c00 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.162+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.162+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94558605212416] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.162+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94558605213568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4400 con 0x5600241f4c00 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.162+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5400 con 0x5600241f4c00 2024-04-27T05:21:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.163+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94558605213568] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.163+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.163+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94558605214720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f4000 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.163+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5c00 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.164+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94558605214720] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.164+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.164+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94558605215872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7000 con 0x5600241f4c00 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=0 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=1 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=2 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=3 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=4 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=5 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=6 2024-04-27T05:21:39.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94558605215872] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 20 add_watcher() i=7 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.165+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da6c00 con 0x560024dd3400 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.166+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.166+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94558605217024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024c4c800 con 0x560024dd3400 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94558605217024] v29'140 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ea3cc9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ec0817880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ec0817880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ec0817880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ec0817880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560023e78780 con 0x560024dd3c00 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ebc4fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ebc4fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59596/0 (socket says 172.21.15.138:59596) 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.167+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1914214203 learned_addr learned my addr 172.21.15.138:0/1914214203 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.168+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1914214203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023ec6680 con 0x560024dd3c00 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.168+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/1914214203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x560024dd4b40 tx=0x5600241aa0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5689f20d9c5ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.168+0000 7f5ea34c8640 1 -- 172.21.15.138:0/1914214203 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5600241f6580 con 0x560024dd3c00 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.168+0000 7f5ea34c8640 1 -- 172.21.15.138:0/1914214203 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560024d28000 con 0x560024dd3c00 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.168+0000 7f5ea34c8640 1 -- 172.21.15.138:0/1914214203 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5600241f66e0 con 0x560024dd3c00 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- 172.21.15.138:0/1914214203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 msgr2=0x560024139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1914214203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x560024dd4b40 tx=0x5600241aa0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- 172.21.15.138:0/1914214203 shutdown_connections 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1914214203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- 172.21.15.138:0/1914214203 >> 172.21.15.138:0/1914214203 conn(0x560024dd3800 msgr2=0x560023f5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- 172.21.15.138:0/1914214203 shutdown_connections 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- 172.21.15.138:0/1914214203 wait complete. 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 Processor -- start 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- start start 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ec0817880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x560023e78500 con 0x560024dd3c00 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ebc4fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ebc4fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59600/0 (socket says 172.21.15.138:59600) 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 learned_addr learned my addr 172.21.15.138:0/2186302907 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.169+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560023ec64e0 con 0x560024dd3c00 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5600241c5770 tx=0x5600241aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ea1cc5640 1 -- 172.21.15.138:0/2186302907 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5600241f6840 con 0x560024dd3c00 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ea1cc5640 1 -- 172.21.15.138:0/2186302907 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560024d28780 con 0x560024dd3c00 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560023e58ea0 con 0x560024dd3c00 2024-04-27T05:21:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ea1cc5640 1 -- 172.21.15.138:0/2186302907 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5600241f6580 con 0x560024dd3c00 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560023e596c0 con 0x560024dd3c00 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.170+0000 7f5ea1cc5640 1 -- 172.21.15.138:0/2186302907 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x560023e7eb00 con 0x560024dd3c00 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ea1cc5640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024209400 0x560024294000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ea1cc5640 1 -- 172.21.15.138:0/2186302907 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x560023e67dc0 con 0x560024dd3c00 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024209400 0x560024294000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd2800 0x560024139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd2400 con 0x560024dd2800 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd2800 0x560024139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024209400 0x560024294000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560024dd4b10 tx=0x5600241aa960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.171+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd2800 0x560024139700 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.172+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'50 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024dd2800 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.172+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4400 0x560024294580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.172+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5400 con 0x5600241f4400 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.172+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4400 0x560024294580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.173+0000 7f5ebc4fa640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4400 0x560024294580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.174+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.174+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4000 0x560024294b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.174+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5c00 con 0x5600241f4000 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.174+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4000 0x560024294b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.174+0000 7f5ebd4fc640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4000 0x560024294b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.175+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.175+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da6800 con 0x560024dd2800 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.176+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'141 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024dd2800 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.176+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560024209800 0x560024295080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.176+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024209c00 con 0x560024209800 2024-04-27T05:21:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.176+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560024209800 0x560024295080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.176+0000 7f5ebccfb640 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560024209800 0x560024295080 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.177+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'44 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.177+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae000 con 0x5600241f4000 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.178+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'40 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.178+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae400 con 0x560024dd2800 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.179+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024dd2800 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.179+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae800 con 0x5600241f4400 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.180+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'31 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.180+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242aec00 con 0x560024209800 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.181+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.181+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af000 con 0x5600241f4000 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.182+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.182+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af400 con 0x560024209800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.183+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'45 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.183+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af800 con 0x5600241f4400 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.184+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.184+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242afc00 con 0x560024209800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.185+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'22 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.185+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae400 con 0x5600241f4400 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.185+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'48 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.186+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242aec00 con 0x560024dd2800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.186+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'142 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024dd2800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.186+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af400 con 0x5600241f4000 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.187+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'38 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.187+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024209c00 con 0x560024209800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.188+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'46 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.188+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7400 con 0x5600241f4000 2024-04-27T05:21:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.189+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'28 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.189+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da6800 con 0x560024dd2800 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.190+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024dd2800 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.190+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024dd2400 con 0x560024dd2800 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.191+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'143 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024dd2800 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.191+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5800 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.192+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'39 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.192+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d6000 con 0x560024209800 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.193+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'32 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.193+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d6400 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.194+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.194+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d6800 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.195+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'34 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.195+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d6c00 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.196+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.196+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7000 con 0x5600241f4000 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.196+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'41 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.196+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7400 con 0x5600241f4400 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.197+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.197+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7800 con 0x5600241f4000 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.198+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'28 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.198+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7c00 con 0x5600241f4400 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.200+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'32 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.200+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7400 con 0x5600241f4000 2024-04-27T05:21:39.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.201+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/2186302907 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'40 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4000 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.201+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d6400 con 0x5600241f4400 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.202+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/2186302907 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'49 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f4400 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.202+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5400 con 0x560024209800 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5600241f2fc0 con 0x560024209800 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5e9f4c0640 20 reqs_thread_entry: start 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ec0817880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ec0817880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5e9d4bc640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5e9cc9a640 20 reqs_thread_entry: start 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ec0817880 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T05:21:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ec0817880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5800 con 0x5600241f4c00 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.203+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 20 rados_obj.operate() r=0 bl.length=385 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7000 con 0x560024dd3400 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 testx$tenanteduser.buckets [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 170+0+15 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T05:21:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.204+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7400 con 0x5600241f4c00 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(64 p5YnriCv1nAtykxBrupQ [delete] v29'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ec0817880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94558605220480 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae800 con 0x5600241f4800 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94558605210112 notify 124554051594 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x560024d6b6c0 con 0x5600241f4800 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5eb74f0640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 94558605210112 notifier 4613 bl.length()=170 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(65 notify.2 [notify cookie 94558605220480 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5600241f3440 con 0x5600241f4800 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5eb74f0640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5400 con 0x5600241f4800 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3440 con 0x5600241f4800 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94558605220480 notify 124554051594 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560024d6b860 con 0x5600241f4800 2024-04-27T05:21:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ec0817880 10 removing email index: 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ec0817880 10 removing user buckets index 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.205+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af800 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.206+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 testx$tenanteduser.buckets [delete] v29'6 uv6 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.206+0000 7f5ec0817880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.206+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [notify cookie 94558605220480 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae400 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94558605212416 notify 124554051599 ret 0) v3 ==== 217+0+0 (crc 0 0 0) 0x560024257860 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5eb74f0640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051599 cookie 94558605212416 notifier 4613 bl.length()=175 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [notify cookie 94558605220480 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5eb74f0640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d6800 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(69 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94558605220480 notify 124554051599 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560024257a00 con 0x560024dd3400 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5ec0817880 10 removing user index: testx$tenanteduser 2024-04-27T05:21:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.207+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242ae000 con 0x5600241f4c00 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 testx$tenanteduser [call,call,delete] v29'8 uv8 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5ec0817880 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5ec0817880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [notify cookie 94558605220480 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af000 con 0x5600241f5000 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== watch-notify(notify (1) cookie 94558605211264 notify 124554051597 ret 0) v3 ==== 209+0+0 (crc 0 0 0) 0x560024257ba0 con 0x5600241f5000 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(71 notify.3 [notify cookie 94558605220480 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5eb6cef640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051597 cookie 94558605211264 notifier 4613 bl.length()=167 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.208+0000 7f5eb6cef640 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7800 con 0x5600241f5000 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.209+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== watch-notify(notify_complete (2) cookie 94558605220480 notify 124554051597 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560024257d40 con 0x5600241f5000 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.209+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(72 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.209+0000 7f5ea3cc9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.209+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94558605217024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af400 con 0x560024dd3400 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.210+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94558605217024] v29'144 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.210+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94558605207808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024209000 con 0x5600241f5000 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.211+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94558605207808] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.211+0000 7f5ec0817880 20 remove_watcher() i=0 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.211+0000 7f5ec0817880 2 removed watcher, disabling cache 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.211+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94558605208960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024208c00 con 0x560024dd3400 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.212+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94558605208960] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.212+0000 7f5ec0817880 20 remove_watcher() i=1 2024-04-27T05:21:39.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.212+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94558605210112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242af800 con 0x5600241f4800 2024-04-27T05:21:39.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.213+0000 7f5ebccfb640 1 -- 172.21.15.138:0/1947523028 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94558605210112] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3440 con 0x5600241f4800 2024-04-27T05:21:39.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.213+0000 7f5ec0817880 20 remove_watcher() i=2 2024-04-27T05:21:39.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.213+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94558605211264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7000 con 0x5600241f5000 2024-04-27T05:21:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.214+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94558605211264] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f3200 con 0x5600241f5000 2024-04-27T05:21:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.214+0000 7f5ec0817880 20 remove_watcher() i=3 2024-04-27T05:21:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.214+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94558605212416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560024da7400 con 0x560024dd3400 2024-04-27T05:21:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.215+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94558605212416] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x560024dd3400 2024-04-27T05:21:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.215+0000 7f5ec0817880 20 remove_watcher() i=4 2024-04-27T05:21:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.215+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94558605213568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600241f5800 con 0x5600241f4c00 2024-04-27T05:21:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.216+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94558605213568] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.216+0000 7f5ec0817880 20 remove_watcher() i=5 2024-04-27T05:21:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.216+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94558605214720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242d7400 con 0x5600241f4c00 2024-04-27T05:21:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.217+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94558605214720] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.217+0000 7f5ec0817880 20 remove_watcher() i=6 2024-04-27T05:21:39.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.217+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94558605215872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600242afc00 con 0x5600241f4c00 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ebd4fc640 1 -- 172.21.15.138:0/1947523028 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94558605215872] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5600241f2900 con 0x5600241f4c00 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 20 remove_watcher() i=7 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4c00 msgr2=0x560024138c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4c00 0x560024138c00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd3400 msgr2=0x560024137600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd3400 0x560024137600 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5600241f5000 msgr2=0x560024136580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5600241f5000 0x560024136580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4800 msgr2=0x560024137b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4800 0x560024137b80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024dd3000 msgr2=0x560024138680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024dd3000 0x560024138680 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x560023f60c00 tx=0x5600241aa120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 msgr2=0x560024137080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 --2- 172.21.15.138:0/1947523028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024c4dc00 0x560024137080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x560024d11980 tx=0x560023f67a40 comp rx=0 tx=0).stop 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 reap_dead start 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ebc4fa640 1 -- 172.21.15.138:0/1947523028 reap_dead start 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 shutdown_connections 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 >> 172.21.15.138:0/1947523028 conn(0x560024c4d800 msgr2=0x560023f5f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 shutdown_connections 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.218+0000 7f5ec0817880 1 -- 172.21.15.138:0/1947523028 wait complete. 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4400 msgr2=0x560024294580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5600241f4400 0x560024294580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd2800 msgr2=0x560024139700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x560024dd2800 0x560024139700 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560024209800 msgr2=0x560024295080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x560024209800 0x560024295080 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4000 msgr2=0x560024294b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5600241f4000 0x560024294b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024209400 msgr2=0x560024294000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024209400 0x560024294000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560024dd4b10 tx=0x5600241aa960 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 msgr2=0x560024139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ebccfb640 1 -- 172.21.15.138:0/2186302907 reap_dead start 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5600241c5770 tx=0x5600241aa000 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 shutdown_connections 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/2186302907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560024dd3c00 0x560024139180 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 >> 172.21.15.138:0/2186302907 conn(0x560024dd3800 msgr2=0x560023f5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 shutdown_connections 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/2186302907 wait complete. 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024da6000 msgr2=0x560024138100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024da6000 0x560024138100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x560024d564e0 tx=0x560023f674a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 msgr2=0x560024136b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136b00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x560024d112f0 tx=0x560023f66c60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 shutdown_connections 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x560024da6000 0x560024138100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 --2- 172.21.15.138:0/3000908217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560023e55400 0x560024136b00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 >> 172.21.15.138:0/3000908217 conn(0x560023e54c00 msgr2=0x560023f5d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 shutdown_connections 2024-04-27T05:21:39.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.219+0000 7f5ec0817880 1 -- 172.21.15.138:0/3000908217 wait complete. 2024-04-27T05:21:39.233 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-27T05:21:39.236 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-04-27T05:21:39.279 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:39.279 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-27T05:21:39.279 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-27T05:21:39.279 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-04-27T05:21:39.323 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.324 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc62f23880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562be47a3400 0x562be4a84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc62f23880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562be4a7a600 con 0x562be47a3000 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc62f23880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x562be47e2280 con 0x562be47a3400 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc5f068640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562be47a3400 0x562be4a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc5f869640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x562be47a3000 0x562be48aa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42880/0 (socket says 172.21.15.138:42880) 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc5f068640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562be47a3400 0x562be4a84580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59608/0 (socket says 172.21.15.138:59608) 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc5f869640 1 -- 172.21.15.138:0/2344400494 learned_addr learned my addr 172.21.15.138:0/2344400494 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc5f068640 1 -- 172.21.15.138:0/2344400494 >> v1:172.21.15.114:6789/0 conn(0x562be47a3000 legacy=0x562be48aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.322+0000 7efc5f068640 1 -- 172.21.15.138:0/2344400494 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be47a7380 con 0x562be47a3400 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.324+0000 7efc5f068640 1 --2- 172.21.15.138:0/2344400494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3400 0x562be4a84580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x562be48aec00 tx=0x562be48b4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77882185b6830597 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.324+0000 7efc5e867640 1 -- 172.21.15.138:0/2344400494 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bbce0 con 0x562be47a3400 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.324+0000 7efc5e867640 1 -- 172.21.15.138:0/2344400494 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be4a7bb00 con 0x562be47a3400 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.324+0000 7efc5e867640 1 -- 172.21.15.138:0/2344400494 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bbe40 con 0x562be47a3400 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- 172.21.15.138:0/2344400494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3400 msgr2=0x562be4a84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 --2- 172.21.15.138:0/2344400494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3400 0x562be4a84580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x562be48aec00 tx=0x562be48b4de0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- 172.21.15.138:0/2344400494 shutdown_connections 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 --2- 172.21.15.138:0/2344400494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3400 0x562be4a84580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- 172.21.15.138:0/2344400494 >> 172.21.15.138:0/2344400494 conn(0x562be47a2c00 msgr2=0x562be48aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- 172.21.15.138:0/2344400494 shutdown_connections 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- 172.21.15.138:0/2344400494 wait complete. 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.325+0000 7efc62f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be47e2780 con 0x562be47a3000 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5f869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5f869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59610/0 (socket says 172.21.15.138:59610) 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5f869640 1 -- 172.21.15.138:0/2816224059 learned_addr learned my addr 172.21.15.138:0/2816224059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5f869640 1 -- 172.21.15.138:0/2816224059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be47a6ea0 con 0x562be47a3000 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5f869640 1 --2- 172.21.15.138:0/2816224059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x562be565ecc0 tx=0x562be48b4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c5d5ba67904a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5e066640 1 -- 172.21.15.138:0/2816224059 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bb4a0 con 0x562be47a3000 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5e066640 1 -- 172.21.15.138:0/2816224059 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be4a7be00 con 0x562be47a3000 2024-04-27T05:21:39.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.326+0000 7efc5e066640 1 -- 172.21.15.138:0/2816224059 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bbce0 con 0x562be47a3000 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- 172.21.15.138:0/2816224059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 msgr2=0x562be4a84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 --2- 172.21.15.138:0/2816224059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x562be565ecc0 tx=0x562be48b4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- 172.21.15.138:0/2816224059 shutdown_connections 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 --2- 172.21.15.138:0/2816224059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- 172.21.15.138:0/2816224059 >> 172.21.15.138:0/2816224059 conn(0x562be47a2c00 msgr2=0x562be48ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- 172.21.15.138:0/2816224059 shutdown_connections 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- 172.21.15.138:0/2816224059 wait complete. 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc62f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be47e28c0 con 0x562be47a3000 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc5f869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc5f869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59618/0 (socket says 172.21.15.138:59618) 2024-04-27T05:21:39.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc5f869640 1 -- 172.21.15.138:0/2544471800 learned_addr learned my addr 172.21.15.138:0/2544471800 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.327+0000 7efc5f869640 1 -- 172.21.15.138:0/2544471800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be47a76c0 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc5f869640 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x562be565eea0 tx=0x562be48b4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc5c863640 1 -- 172.21.15.138:0/2544471800 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bbe40 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc5c863640 1 -- 172.21.15.138:0/2544471800 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be567a000 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562be47a71e0 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc5c863640 1 -- 172.21.15.138:0/2544471800 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bb4a0 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562be4814ea0 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc5c863640 1 -- 172.21.15.138:0/2544471800 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562be47cd600 con 0x562be47a3000 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc62f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.328+0000 7efc62f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be47c6280 con 0x562be559b800 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5c863640 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be56fe000 0x562be4a85080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5c863640 1 -- 172.21.15.138:0/2544471800 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x562be47b5a40 con 0x562be47a3000 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc6006a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc6006a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59632/0 (socket says 172.21.15.138:59632) 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc6006a640 1 -- 172.21.15.138:0/1500511145 learned_addr learned my addr 172.21.15.138:0/1500511145 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5f869640 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be56fe000 0x562be4a85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc6006a640 1 -- 172.21.15.138:0/1500511145 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be4814d00 con 0x562be559b800 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5f869640 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be56fe000 0x562be4a85080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x562be56ae450 tx=0x562be48b54a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc6006a640 1 --2- 172.21.15.138:0/1500511145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x562be565ff50 tx=0x562be48b4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af1139746b69dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5a05e640 1 -- 172.21.15.138:0/1500511145 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bbe40 con 0x562be559b800 2024-04-27T05:21:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5a05e640 1 -- 172.21.15.138:0/1500511145 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be567a300 con 0x562be559b800 2024-04-27T05:21:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.329+0000 7efc5a05e640 1 -- 172.21.15.138:0/1500511145 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bb4a0 con 0x562be559b800 2024-04-27T05:21:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- 172.21.15.138:0/1500511145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 msgr2=0x562be4a84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 --2- 172.21.15.138:0/1500511145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x562be565ff50 tx=0x562be48b4c00 comp rx=0 tx=0).stop 2024-04-27T05:21:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- 172.21.15.138:0/1500511145 shutdown_connections 2024-04-27T05:21:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 --2- 172.21.15.138:0/1500511145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- 172.21.15.138:0/1500511145 >> 172.21.15.138:0/1500511145 conn(0x562be47a3400 msgr2=0x562be48ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- 172.21.15.138:0/1500511145 shutdown_connections 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- 172.21.15.138:0/1500511145 wait complete. 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc62f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be47c7180 con 0x562be559b800 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc6006a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc6006a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59648/0 (socket says 172.21.15.138:59648) 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 learned_addr learned my addr 172.21.15.138:0/1329099103 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be4814b60 con 0x562be559b800 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.330+0000 7efc6006a640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x562be565f950 tx=0x562be48b51a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc5885b640 1 -- 172.21.15.138:0/1329099103 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bbe40 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc5885b640 1 -- 172.21.15.138:0/1329099103 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be567a480 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562be48149c0 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc5885b640 1 -- 172.21.15.138:0/1329099103 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be47bb4a0 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562be4814820 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc5885b640 1 -- 172.21.15.138:0/1329099103 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562be47cd8c0 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc5885b640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be4b26000 0x562be4a85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc5885b640 1 -- 172.21.15.138:0/1329099103 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x562be47b5c00 con 0x562be559b800 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.331+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc5f068640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be4b26000 0x562be4a85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559bc00 0x562be4a85b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26400 con 0x562be559bc00 2024-04-27T05:21:39.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc6006a640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559bc00 0x562be4a85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc5f068640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be4b26000 0x562be4a85600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x562be48aec00 tx=0x562be4aee120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc6006a640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559bc00 0x562be4a85b80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 20 realm 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26800 con 0x562be559bc00 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.332+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26c00 con 0x562be559bc00 2024-04-27T05:21:39.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27000 con 0x562be559bc00 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4b27400 0x562be4a86100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26400 con 0x562be4b27400 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc5f869640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4b27400 0x562be4a86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.333+0000 7efc5f869640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4b27400 0x562be4a86100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc62f23880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc62f23880 20 searching for the correct realm 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26800 con 0x562be4b27400 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc5f869640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4b26800 0x562be4a86c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26400 con 0x562be4b26800 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc5f068640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4b26800 0x562be4a86c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.334+0000 7efc5f068640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4b26800 0x562be4a86c00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc5f068640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4b26400 0x562be4a87180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc6006a640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4b26400 0x562be4a87180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc6006a640 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4b26400 0x562be4a87180 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.335+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.336+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26800 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.337+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.337+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.337+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.337+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.337+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.337+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26800 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.338+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.339+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26800 2024-04-27T05:21:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.340+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.341+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.341+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.341+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.341+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.341+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.341+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26800 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b27400 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26400 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be559bc00 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.342+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be559a800 con 0x562be4b26800 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26c00 con 0x562be559bc00 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27000 con 0x562be559bc00 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27800 con 0x562be4b27400 2024-04-27T05:21:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 zone default found 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 4 Realm: () 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 zonegroup default 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.343+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27c00 con 0x562be4b26400 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586000 con 0x562be4b27400 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586400 con 0x562be4b27400 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 started sync module instance, tier type = 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.344+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586800 con 0x562be4b26800 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.345+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.345+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94746519783680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586c00 con 0x562be4b26800 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.345+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5587000 con 0x562be559bc00 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.346+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94746519783680] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.346+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.346+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94746519784832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5587400 con 0x562be559bc00 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.346+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5587800 con 0x562be4b27400 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.347+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94746519784832] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.348+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.348+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94746519785984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586000 con 0x562be4b27400 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.348+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586400 con 0x562be4b26800 2024-04-27T05:21:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.349+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94746519785984] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.349+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.349+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94746519787136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5587800 con 0x562be4b26800 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.349+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27800 con 0x562be559bc00 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.350+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.350+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94746519787136] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.350+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94746519788288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b26c00 con 0x562be559bc00 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.350+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27000 con 0x562be4b26400 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.351+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94746519788288] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.351+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.351+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94746519789440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27c00 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.351+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be56ff800 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.352+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94746519789440] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.352+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.352+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94746519790592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586800 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.352+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586c00 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94746519790592] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94746519791744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586400 con 0x562be4b26400 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=0 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=1 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=2 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=3 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=4 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=5 2024-04-27T05:21:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.353+0000 7efc62f23880 20 add_watcher() i=6 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.354+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94746519791744] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.354+0000 7efc62f23880 20 add_watcher() i=7 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.354+0000 7efc62f23880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.354+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5587800 con 0x562be559bc00 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.354+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.354+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94746519792896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5587000 con 0x562be559bc00 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94746519792896] v29'145 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc46837640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc62f23880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc62f23880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc62f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.355+0000 7efc62f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be47c6780 con 0x562be5587c00 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc5f068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc5f068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59664/0 (socket says 172.21.15.138:59664) 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc5f068640 1 -- 172.21.15.138:0/420561770 learned_addr learned my addr 172.21.15.138:0/420561770 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc5f068640 1 -- 172.21.15.138:0/420561770 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be4814680 con 0x562be5587c00 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc5f068640 1 --2- 172.21.15.138:0/420561770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x562be5718780 tx=0x562be4aee0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3323f1aa2d0e90e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc46036640 1 -- 172.21.15.138:0/420561770 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be5588580 con 0x562be5587c00 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc46036640 1 -- 172.21.15.138:0/420561770 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be567a000 con 0x562be5587c00 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.356+0000 7efc46036640 1 -- 172.21.15.138:0/420561770 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be55886e0 con 0x562be5587c00 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 -- 172.21.15.138:0/420561770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 msgr2=0x562be4a86680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 --2- 172.21.15.138:0/420561770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x562be5718780 tx=0x562be4aee0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 -- 172.21.15.138:0/420561770 shutdown_connections 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 --2- 172.21.15.138:0/420561770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 -- 172.21.15.138:0/420561770 >> 172.21.15.138:0/420561770 conn(0x562be5587400 msgr2=0x562be48ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 -- 172.21.15.138:0/420561770 shutdown_connections 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 -- 172.21.15.138:0/420561770 wait complete. 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.357+0000 7efc62f23880 1 Processor -- start 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc62f23880 1 -- start start 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc62f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc62f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562be47c6500 con 0x562be5587c00 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc5f068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc5f068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59676/0 (socket says 172.21.15.138:59676) 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 learned_addr learned my addr 172.21.15.138:0/1982805079 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562be48144e0 con 0x562be5587c00 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc5f068640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x562be4b09740 tx=0x562be4aee000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc44833640 1 -- 172.21.15.138:0/1982805079 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be5588840 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc44833640 1 -- 172.21.15.138:0/1982805079 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562be567a780 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562be47a6ea0 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc44833640 1 -- 172.21.15.138:0/1982805079 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562be5588580 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.358+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562be47a76c0 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc44833640 1 -- 172.21.15.138:0/1982805079 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562be47ccb00 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc44833640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be5586800 0x562be4a87700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc44833640 1 -- 172.21.15.138:0/1982805079 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x562be47b5dc0 con 0x562be5587c00 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc6006a640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be5586800 0x562be4a87700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559a800 0x562be4be2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be6000 con 0x562be559a800 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.359+0000 7efc5f869640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559a800 0x562be4be2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.360+0000 7efc6006a640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be5586800 0x562be4a87700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x562be5718ba0 tx=0x562be4aee960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.360+0000 7efc5f869640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559a800 0x562be4be2000 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.361+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v29'51 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be559a800 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.361+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4be6400 0x562be4be2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.361+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be6800 con 0x562be4be6400 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.361+0000 7efc5f068640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4be6400 0x562be4be2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.361+0000 7efc5f068640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4be6400 0x562be4be2580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.362+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v29'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.362+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4be6c00 0x562be4be2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.362+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7000 con 0x562be4be6c00 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.362+0000 7efc6006a640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4be6c00 0x562be4be2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.363+0000 7efc6006a640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4be6c00 0x562be4be2b00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.363+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v29'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.363+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7400 con 0x562be559a800 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.364+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v29'146 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be559a800 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.364+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4be7800 0x562be4be3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.364+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7c00 con 0x562be4be7800 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.364+0000 7efc5f869640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4be7800 0x562be4be3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.365+0000 7efc5f869640 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4be7800 0x562be4be3080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.366+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v29'47 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.366+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e000 con 0x562be4be6c00 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.366+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v29'42 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.366+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e400 con 0x562be559a800 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.367+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v29'26 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be559a800 2024-04-27T05:21:39.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.367+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e800 con 0x562be4be6400 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.368+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v29'33 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.368+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0ec00 con 0x562be4be7800 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.369+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v29'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.369+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f000 con 0x562be4be6c00 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.370+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v29'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.370+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f400 con 0x562be4be7800 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.371+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v29'48 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.371+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f800 con 0x562be4be6400 2024-04-27T05:21:39.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.372+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v29'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.372+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0fc00 con 0x562be4be7800 2024-04-27T05:21:39.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.373+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v29'23 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.373+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e400 con 0x562be4be6400 2024-04-27T05:21:39.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.374+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v29'50 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.374+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0ec00 con 0x562be559a800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.375+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v29'147 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be559a800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.375+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be6000 con 0x562be4be6c00 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.376+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v29'41 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.376+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7400 con 0x562be4be7800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.376+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v29'49 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.376+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7c00 con 0x562be4be6c00 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.377+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v29'29 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.378+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586000 con 0x562be559a800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.379+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v29'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be559a800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.379+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7000 con 0x562be559a800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.379+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v29'148 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be559a800 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.379+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be6000 con 0x562be4be6c00 2024-04-27T05:21:39.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.380+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v29'42 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.380+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7c00 con 0x562be4be7800 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.381+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v29'33 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.381+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e000 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.382+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v29'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.382+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f000 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.383+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v29'35 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.383+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586c00 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.384+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v29'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.384+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586400 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.385+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v29'43 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.385+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7400 con 0x562be4be6400 2024-04-27T05:21:39.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.386+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v29'26 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.386+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7000 con 0x562be4be6c00 2024-04-27T05:21:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.387+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v29'29 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.387+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7c00 con 0x562be4be6400 2024-04-27T05:21:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.388+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v29'34 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.388+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586000 con 0x562be4be6c00 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.389+0000 7efc6006a640 1 -- 172.21.15.138:0/1982805079 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v29'43 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4be6c00 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.389+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f400 con 0x562be4be6400 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.391+0000 7efc5f068640 1 -- 172.21.15.138:0/1982805079 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v29'51 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4be6400 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.391+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0fc00 con 0x562be4be7800 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v29'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4be7800 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc4202e640 20 reqs_thread_entry: start 2024-04-27T05:21:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc62f23880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc62f23880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc4002a640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc3f808640 20 reqs_thread_entry: start 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc62f23880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be6800 con 0x562be4b26800 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.392+0000 7efc62f23880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7400 con 0x562be4b26800 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 get_obj_state: rctx=0x562be4c4e230 obj=myfoo:four state=0x562be4c6a1e8 s->prefetch_data=0 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:64 3.8 3:117421df:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e800 con 0x562be4b27400 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(64 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1_four [getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 234+0+865 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 10 manifest: total_size = 4 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.4584667501323926646 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 get_obj_state: rctx=0x562be4c4e230 obj=myfoo:four state=0x562be4c6a1e8 s->prefetch_data=0 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 prepare_atomic_modification: state is not atomic. state=0x562be4c6a1e8 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.393+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f800 con 0x562be559bc00 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.394+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(65 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [stat,call,call] v29'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.395+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 3.8 3:117421df:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e400 con 0x562be4b27400 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.395+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(66 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1_four [call] v29'2 uv2 ondisk = 0) v8 ==== 192+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.395+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4b27000 con 0x562be559bc00 2024-04-27T05:21:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.396+0000 7efc46837640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.396+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94746519792896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0e000 con 0x562be559bc00 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.397+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [stat,call,call] v29'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.397+0000 7efc5a05e640 20 handle_completion(): completion ok for obj=four 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.397+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 data_loggenerations_metadata [watch unwatch cookie 94746519792896] v29'149 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.397+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94746519783680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f000 con 0x562be4b26800 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.398+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(69 notify.0 [watch unwatch cookie 94746519783680] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.398+0000 7efc62f23880 20 remove_watcher() i=0 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.398+0000 7efc62f23880 2 removed watcher, disabling cache 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.398+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94746519784832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4c0f800 con 0x562be559bc00 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.399+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(70 notify.1 [watch unwatch cookie 94746519784832] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.399+0000 7efc62f23880 20 remove_watcher() i=1 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.399+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94746519785984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586c00 con 0x562be4b27400 2024-04-27T05:21:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.400+0000 7efc5f869640 1 -- 172.21.15.138:0/1329099103 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(71 notify.2 [watch unwatch cookie 94746519785984] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3cfc0 con 0x562be4b27400 2024-04-27T05:21:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.400+0000 7efc62f23880 20 remove_watcher() i=2 2024-04-27T05:21:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.400+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94746519787136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586400 con 0x562be4b26800 2024-04-27T05:21:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.401+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(72 notify.3 [watch unwatch cookie 94746519787136] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3d200 con 0x562be4b26800 2024-04-27T05:21:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.401+0000 7efc62f23880 20 remove_watcher() i=3 2024-04-27T05:21:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.401+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94746519788288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be5586000 con 0x562be559bc00 2024-04-27T05:21:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.402+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 notify.4 [watch unwatch cookie 94746519788288] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be559bc00 2024-04-27T05:21:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.402+0000 7efc62f23880 20 remove_watcher() i=4 2024-04-27T05:21:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.402+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94746519789440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be6000 con 0x562be4b26400 2024-04-27T05:21:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.403+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(74 notify.5 [watch unwatch cookie 94746519789440] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.403+0000 7efc62f23880 20 remove_watcher() i=5 2024-04-27T05:21:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.403+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94746519790592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4be7000 con 0x562be4b26400 2024-04-27T05:21:39.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.404+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(75 notify.6 [watch unwatch cookie 94746519790592] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.404+0000 7efc62f23880 20 remove_watcher() i=6 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.404+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94746519791744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x562be4ad0000 con 0x562be4b26400 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc6006a640 1 -- 172.21.15.138:0/1329099103 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(76 notify.7 [watch unwatch cookie 94746519791744] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562be4b3c900 con 0x562be4b26400 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 20 remove_watcher() i=7 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4b26400 msgr2=0x562be4a87180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4b26400 0x562be4a87180 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559bc00 msgr2=0x562be4a85b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559bc00 0x562be4a85b80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4b26800 msgr2=0x562be4a86c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4b26800 0x562be4a86c00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4b27400 msgr2=0x562be4a86100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4b27400 0x562be4a86100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be4b26000 msgr2=0x562be4a85600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be4b26000 0x562be4a85600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x562be48aec00 tx=0x562be4aee120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 msgr2=0x562be4a84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1329099103 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be559b800 0x562be4a84b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x562be565f950 tx=0x562be48b51a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 reap_dead start 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc5f068640 1 -- 172.21.15.138:0/1329099103 reap_dead start 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 shutdown_connections 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 >> 172.21.15.138:0/1329099103 conn(0x562be47a3400 msgr2=0x562be48ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 shutdown_connections 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1329099103 wait complete. 2024-04-27T05:21:39.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4be6400 msgr2=0x562be4be2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562be4be6400 0x562be4be2580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559a800 msgr2=0x562be4be2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562be559a800 0x562be4be2000 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4be7800 msgr2=0x562be4be3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562be4be7800 0x562be4be3080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4be6c00 msgr2=0x562be4be2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562be4be6c00 0x562be4be2b00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be5586800 msgr2=0x562be4a87700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be5586800 0x562be4a87700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x562be5718ba0 tx=0x562be4aee960 comp rx=0 tx=0).stop 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 msgr2=0x562be4a86680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc5f869640 1 -- 172.21.15.138:0/1982805079 reap_dead start 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.405+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x562be4b09740 tx=0x562be4aee000 comp rx=0 tx=0).stop 2024-04-27T05:21:39.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 shutdown_connections 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 --2- 172.21.15.138:0/1982805079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be5587c00 0x562be4a86680 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 >> 172.21.15.138:0/1982805079 conn(0x562be5587400 msgr2=0x562be48ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 shutdown_connections 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/1982805079 wait complete. 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be56fe000 msgr2=0x562be4a85080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be56fe000 0x562be4a85080 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x562be56ae450 tx=0x562be48b54a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 msgr2=0x562be4a84580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x562be565eea0 tx=0x562be48b4c60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 shutdown_connections 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562be56fe000 0x562be4a85080 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 --2- 172.21.15.138:0/2544471800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562be47a3000 0x562be4a84580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 >> 172.21.15.138:0/2544471800 conn(0x562be47a2c00 msgr2=0x562be48ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 shutdown_connections 2024-04-27T05:21:39.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.406+0000 7efc62f23880 1 -- 172.21.15.138:0/2544471800 wait complete. 2024-04-27T05:21:39.415 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T05:21:39.415 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T05:21:39.415 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T05:21:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.455+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.455+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.455+0000 7f3c846bb880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5577cfeaf400 0x5577d0190580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c846bb880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5577d0186600 con 0x5577cfeaf000 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c846bb880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5577cfeee280 con 0x5577cfeaf400 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c8265f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5577cfeaf400 0x5577d0190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c82e60640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5577cfeaf000 0x5577cffb6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42884/0 (socket says 172.21.15.138:42884) 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c82e60640 1 -- 172.21.15.138:0/1190435920 learned_addr learned my addr 172.21.15.138:0/1190435920 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c8265f640 1 -- 172.21.15.138:0/1190435920 >> v1:172.21.15.114:6789/0 conn(0x5577cfeaf000 legacy=0x5577cffb6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.456+0000 7f3c8265f640 1 -- 172.21.15.138:0/1190435920 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cfeb3380 con 0x5577cfeaf400 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c8265f640 1 --2- 172.21.15.138:0/1190435920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf400 0x5577d0190580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x5577cffbac00 tx=0x5577cffc0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c7f5d1f698df30 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c81e5e640 1 -- 172.21.15.138:0/1190435920 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec7ce0 con 0x5577cfeaf400 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c81e5e640 1 -- 172.21.15.138:0/1190435920 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0187b00 con 0x5577cfeaf400 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c81e5e640 1 -- 172.21.15.138:0/1190435920 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec7e40 con 0x5577cfeaf400 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 -- 172.21.15.138:0/1190435920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf400 msgr2=0x5577d0190580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 --2- 172.21.15.138:0/1190435920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf400 0x5577d0190580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x5577cffbac00 tx=0x5577cffc0de0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 -- 172.21.15.138:0/1190435920 shutdown_connections 2024-04-27T05:21:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 --2- 172.21.15.138:0/1190435920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf400 0x5577d0190580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 -- 172.21.15.138:0/1190435920 >> 172.21.15.138:0/1190435920 conn(0x5577cfeaec00 msgr2=0x5577cffb6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 -- 172.21.15.138:0/1190435920 shutdown_connections 2024-04-27T05:21:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.458+0000 7f3c846bb880 1 -- 172.21.15.138:0/1190435920 wait complete. 2024-04-27T05:21:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c846bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c846bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cfeee780 con 0x5577cfeaf000 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c82e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c82e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59700/0 (socket says 172.21.15.138:59700) 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.459+0000 7f3c82e60640 1 -- 172.21.15.138:0/2900195715 learned_addr learned my addr 172.21.15.138:0/2900195715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c82e60640 1 -- 172.21.15.138:0/2900195715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cfeb2ea0 con 0x5577cfeaf000 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c82e60640 1 --2- 172.21.15.138:0/2900195715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5577d0d6acc0 tx=0x5577cffc0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1a2ae8b98996f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c8165d640 1 -- 172.21.15.138:0/2900195715 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec74a0 con 0x5577cfeaf000 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c8165d640 1 -- 172.21.15.138:0/2900195715 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0187e00 con 0x5577cfeaf000 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c8165d640 1 -- 172.21.15.138:0/2900195715 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec7ce0 con 0x5577cfeaf000 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 -- 172.21.15.138:0/2900195715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 msgr2=0x5577d0190580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2900195715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5577d0d6acc0 tx=0x5577cffc0cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 -- 172.21.15.138:0/2900195715 shutdown_connections 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2900195715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 -- 172.21.15.138:0/2900195715 >> 172.21.15.138:0/2900195715 conn(0x5577cfeaec00 msgr2=0x5577cffb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 -- 172.21.15.138:0/2900195715 shutdown_connections 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.460+0000 7f3c846bb880 1 -- 172.21.15.138:0/2900195715 wait complete. 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c846bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c846bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cfeee8c0 con 0x5577cfeaf000 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c82e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c82e60640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59716/0 (socket says 172.21.15.138:59716) 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c82e60640 1 -- 172.21.15.138:0/225864909 learned_addr learned my addr 172.21.15.138:0/225864909 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c82e60640 1 -- 172.21.15.138:0/225864909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cfeb36c0 con 0x5577cfeaf000 2024-04-27T05:21:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c82e60640 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5577d0d6aea0 tx=0x5577cffc0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c7fe5a640 1 -- 172.21.15.138:0/225864909 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec7e40 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c7fe5a640 1 -- 172.21.15.138:0/225864909 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0d80000 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.461+0000 7f3c7fe5a640 1 -- 172.21.15.138:0/225864909 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec74a0 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5577cfeb31e0 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577cff20ea0 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c7fe5a640 1 -- 172.21.15.138:0/225864909 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5577cfed9600 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c846bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c846bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cfeeec80 con 0x5577d0ca7800 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c7fe5a640 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0e04000 0x5577d0191080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c7fe5a640 1 -- 172.21.15.138:0/225864909 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5577cfec1a40 con 0x5577cfeaf000 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c83661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c83661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59726/0 (socket says 172.21.15.138:59726) 2024-04-27T05:21:39.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.462+0000 7f3c83661640 1 -- 172.21.15.138:0/1629988096 learned_addr learned my addr 172.21.15.138:0/1629988096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c82e60640 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0e04000 0x5577d0191080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c83661640 1 -- 172.21.15.138:0/1629988096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cff20d00 con 0x5577d0ca7800 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c82e60640 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0e04000 0x5577d0191080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5577d0db4450 tx=0x5577cffc14a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c83661640 1 --2- 172.21.15.138:0/1629988096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5577d0d6bec0 tx=0x5577cffc0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba0454d1fcd2c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c7d655640 1 -- 172.21.15.138:0/1629988096 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec7e40 con 0x5577d0ca7800 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c7d655640 1 -- 172.21.15.138:0/1629988096 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0d80300 con 0x5577d0ca7800 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c7d655640 1 -- 172.21.15.138:0/1629988096 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec74a0 con 0x5577d0ca7800 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 -- 172.21.15.138:0/1629988096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 msgr2=0x5577d0190b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 --2- 172.21.15.138:0/1629988096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5577d0d6bec0 tx=0x5577cffc0c00 comp rx=0 tx=0).stop 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 -- 172.21.15.138:0/1629988096 shutdown_connections 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 --2- 172.21.15.138:0/1629988096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 -- 172.21.15.138:0/1629988096 >> 172.21.15.138:0/1629988096 conn(0x5577cfeaf400 msgr2=0x5577cffb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 -- 172.21.15.138:0/1629988096 shutdown_connections 2024-04-27T05:21:39.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.463+0000 7f3c846bb880 1 -- 172.21.15.138:0/1629988096 wait complete. 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c846bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c846bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cfed2280 con 0x5577d0ca7800 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c83661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c83661640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59738/0 (socket says 172.21.15.138:59738) 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 learned_addr learned my addr 172.21.15.138:0/3498851231 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cff20b60 con 0x5577d0ca7800 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c83661640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5577d0cb3470 tx=0x5577cffc11a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.464+0000 7f3c7be52640 1 -- 172.21.15.138:0/3498851231 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec7e40 con 0x5577d0ca7800 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c7be52640 1 -- 172.21.15.138:0/3498851231 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0d80480 con 0x5577d0ca7800 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5577cff209c0 con 0x5577d0ca7800 2024-04-27T05:21:39.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c7be52640 1 -- 172.21.15.138:0/3498851231 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577cfec74a0 con 0x5577d0ca7800 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577cff20820 con 0x5577d0ca7800 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c7be52640 1 -- 172.21.15.138:0/3498851231 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5577cfed98c0 con 0x5577d0ca7800 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c7be52640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0232000 0x5577d0191600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c7be52640 1 -- 172.21.15.138:0/3498851231 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5577cfec1c00 con 0x5577d0ca7800 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.465+0000 7f3c8265f640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0232000 0x5577d0191600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5577d0ca7c00 0x5577d0191b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232400 con 0x5577d0ca7c00 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c83661640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5577d0ca7c00 0x5577d0191b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c8265f640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0232000 0x5577d0191600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5577cffbac00 tx=0x5577d01fc120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c83661640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5577d0ca7c00 0x5577d0191b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 20 realm 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232800 con 0x5577d0ca7c00 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.466+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232c00 con 0x5577d0ca7c00 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233000 con 0x5577d0ca7c00 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5577d0233400 0x5577d0192100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232400 con 0x5577d0233400 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c82e60640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5577d0233400 0x5577d0192100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c82e60640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5577d0233400 0x5577d0192100 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 20 searching for the correct realm 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.467+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232800 con 0x5577d0233400 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c82e60640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5577d0232800 0x5577d0192c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232400 con 0x5577d0232800 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c8265f640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5577d0232800 0x5577d0192c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c8265f640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5577d0232800 0x5577d0192c00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c8265f640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5577d0232400 0x5577d0193180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.468+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.469+0000 7f3c83661640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5577d0232400 0x5577d0193180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.469+0000 7f3c83661640 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5577d0232400 0x5577d0193180 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.469+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.469+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.469+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.469+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232800 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.470+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232800 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.471+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.472+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.473+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232800 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.474+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232800 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0233400 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232400 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.475+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0ca7c00 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0ca6800 con 0x5577d0232800 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232c00 con 0x5577d0ca7c00 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233000 con 0x5577d0ca7c00 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233800 con 0x5577d0233400 2024-04-27T05:21:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 zone default found 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 4 Realm: () 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 zonegroup default 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.476+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233c00 con 0x5577d0232400 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92000 con 0x5577d0233400 2024-04-27T05:21:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92400 con 0x5577d0233400 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 started sync module instance, tier type = 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.477+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92800 con 0x5577d0232800 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.478+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.478+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93973080738048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92c00 con 0x5577d0232800 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.478+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c93000 con 0x5577d0ca7c00 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.479+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93973080738048] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.479+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.480+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93973080739200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c93400 con 0x5577d0ca7c00 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.480+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c93800 con 0x5577d0233400 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.481+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93973080739200] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.481+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.481+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93973080740352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92000 con 0x5577d0233400 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.481+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92400 con 0x5577d0232800 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.482+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93973080740352] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.482+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.482+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93973080741504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c93800 con 0x5577d0232800 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.482+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233800 con 0x5577d0ca7c00 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.483+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.483+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93973080742656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0232c00 con 0x5577d0ca7c00 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.483+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233000 con 0x5577d0232400 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.483+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93973080741504] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.484+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.484+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93973080743808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233c00 con 0x5577d0232400 2024-04-27T05:21:39.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.484+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0e05800 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.484+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93973080742656] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.485+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93973080743808] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.485+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.485+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93973080744960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92800 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.485+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92c00 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93973080744960] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93973080746112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92400 con 0x5577d0232400 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=0 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=1 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=2 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=3 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=4 2024-04-27T05:21:39.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=5 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.486+0000 7f3c846bb880 20 add_watcher() i=6 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.487+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93973080746112] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.487+0000 7f3c846bb880 20 add_watcher() i=7 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.487+0000 7f3c846bb880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.487+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c93800 con 0x5577d0ca7c00 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.487+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.487+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93973080747264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c93000 con 0x5577d0ca7c00 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93973080747264] v29'150 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c69e2e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c846bb880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c846bb880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c846bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.488+0000 7f3c846bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cfed28c0 con 0x5577d0c93c00 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c8265f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c8265f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59740/0 (socket says 172.21.15.138:59740) 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c8265f640 1 -- 172.21.15.138:0/2959584217 learned_addr learned my addr 172.21.15.138:0/2959584217 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c8265f640 1 -- 172.21.15.138:0/2959584217 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cff20680 con 0x5577d0c93c00 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c8265f640 1 --2- 172.21.15.138:0/2959584217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5577d0e24780 tx=0x5577d01fc0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306eb48d26650cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c6962d640 1 -- 172.21.15.138:0/2959584217 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577d0c94580 con 0x5577d0c93c00 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c6962d640 1 -- 172.21.15.138:0/2959584217 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0d80000 con 0x5577d0c93c00 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.489+0000 7f3c6962d640 1 -- 172.21.15.138:0/2959584217 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577d0c946e0 con 0x5577d0c93c00 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- 172.21.15.138:0/2959584217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 msgr2=0x5577d0192680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2959584217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5577d0e24780 tx=0x5577d01fc0c0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- 172.21.15.138:0/2959584217 shutdown_connections 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2959584217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- 172.21.15.138:0/2959584217 >> 172.21.15.138:0/2959584217 conn(0x5577d0c93400 msgr2=0x5577cffb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- 172.21.15.138:0/2959584217 shutdown_connections 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- 172.21.15.138:0/2959584217 wait complete. 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 Processor -- start 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- start start 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.490+0000 7f3c846bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577cfed2780 con 0x5577d0c93c00 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c8265f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c8265f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59742/0 (socket says 172.21.15.138:59742) 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c8265f640 1 -- 172.21.15.138:0/2874212508 learned_addr learned my addr 172.21.15.138:0/2874212508 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c8265f640 1 -- 172.21.15.138:0/2874212508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577cff204e0 con 0x5577d0c93c00 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c8265f640 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5577d0217740 tx=0x5577d01fc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c67e2a640 1 -- 172.21.15.138:0/2874212508 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577d0c94840 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c67e2a640 1 -- 172.21.15.138:0/2874212508 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5577d0d80780 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5577cfeb2ea0 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c67e2a640 1 -- 172.21.15.138:0/2874212508 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5577d0c94580 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.491+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577cfeb36c0 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c67e2a640 1 -- 172.21.15.138:0/2874212508 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5577cfed8b00 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c67e2a640 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0c92800 0x5577d0193700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c67e2a640 1 -- 172.21.15.138:0/2874212508 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5577cfec1dc0 con 0x5577d0c93c00 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c846bb880 5 note: GC not initialized 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c83661640 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0c92800 0x5577d0193700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c65625640 20 reqs_thread_entry: start 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.492+0000 7f3c846bb880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c83661640 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0c92800 0x5577d0193700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5577d0e24bd0 tx=0x5577d01fc960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c63621640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c62dff640 20 reqs_thread_entry: start 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324000 con 0x5577d0232800 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324400 con 0x5577d0232800 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:39.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.493+0000 7f3c846bb880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324800 con 0x5577d0232800 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324c00 con 0x5577d0232800 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(65 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 4.34 4:2e3daf19:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325000 con 0x5577d0232800 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.35 4:adc4a58e:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325400 con 0x5577d0ca7c00 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 4.2c 4:346a3ea0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325800 con 0x5577d0ca7c00 2024-04-27T05:21:39.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 4.2e 4:74cdb6e4:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325c00 con 0x5577d0233400 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 4.3a 4:5decd619:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324000 con 0x5577d0233400 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.494+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 4.c 4:335b21e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324400 con 0x5577d0ca7c00 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.9 4:92e31338:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324800 con 0x5577d0ca7c00 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.2f 4:f6a773d6:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324c00 con 0x5577d0ca7c00 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.2 4:4123e525:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325000 con 0x5577d0ca7c00 2024-04-27T05:21:39.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 4.3e 4:7d356189:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233c00 con 0x5577d0232800 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 4.16 4:6823909f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0233000 con 0x5577d0233400 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.5 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.6 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.495+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(76 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.496+0000 7f3c69e2e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.496+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93973080747264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325400 con 0x5577d0ca7c00 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.497+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 93973080747264] v29'151 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.497+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93973080738048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0325800 con 0x5577d0232800 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.498+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 93973080738048] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.498+0000 7f3c846bb880 20 remove_watcher() i=0 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.498+0000 7f3c846bb880 2 removed watcher, disabling cache 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.498+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93973080739200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324400 con 0x5577d0ca7c00 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.499+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 93973080739200] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.499+0000 7f3c846bb880 20 remove_watcher() i=1 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.499+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93973080740352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324800 con 0x5577d0233400 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.500+0000 7f3c82e60640 1 -- 172.21.15.138:0/3498851231 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 93973080740352] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248fc0 con 0x5577d0233400 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.500+0000 7f3c846bb880 20 remove_watcher() i=2 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.500+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93973080741504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0324c00 con 0x5577d0232800 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.501+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 93973080741504] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0249200 con 0x5577d0232800 2024-04-27T05:21:39.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.501+0000 7f3c846bb880 20 remove_watcher() i=3 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.501+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93973080742656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92c00 con 0x5577d0ca7c00 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.502+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 93973080742656] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0ca7c00 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.502+0000 7f3c846bb880 20 remove_watcher() i=4 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.502+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93973080743808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0c92400 con 0x5577d0232400 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.503+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 93973080743808] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.503+0000 7f3c846bb880 20 remove_watcher() i=5 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.503+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93973080744960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0e0c000 con 0x5577d0232400 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.504+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 93973080744960] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.504+0000 7f3c846bb880 20 remove_watcher() i=6 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.504+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93973080746112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5577d0e0c400 con 0x5577d0232400 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c83661640 1 -- 172.21.15.138:0/3498851231 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 93973080746112] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5577d0248900 con 0x5577d0232400 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 20 remove_watcher() i=7 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5577d0232400 msgr2=0x5577d0193180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5577d0232400 0x5577d0193180 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5577d0ca7c00 msgr2=0x5577d0191b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5577d0ca7c00 0x5577d0191b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5577d0232800 msgr2=0x5577d0192c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5577d0232800 0x5577d0192c00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5577d0233400 msgr2=0x5577d0192100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5577d0233400 0x5577d0192100 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0232000 msgr2=0x5577d0191600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0232000 0x5577d0191600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5577cffbac00 tx=0x5577d01fc120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 msgr2=0x5577d0190b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5577d0cb3470 tx=0x5577cffc11a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c8265f640 1 -- 172.21.15.138:0/3498851231 reap_dead start 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 shutdown_connections 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/3498851231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0ca7800 0x5577d0190b00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 >> 172.21.15.138:0/3498851231 conn(0x5577cfeaf400 msgr2=0x5577cffb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 shutdown_connections 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/3498851231 wait complete. 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0c92800 msgr2=0x5577d0193700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0c92800 0x5577d0193700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5577d0e24bd0 tx=0x5577d01fc960 comp rx=0 tx=0).stop 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 msgr2=0x5577d0192680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.505+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5577d0217740 tx=0x5577d01fc000 comp rx=0 tx=0).stop 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 shutdown_connections 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0c92800 0x5577d0193700 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 --2- 172.21.15.138:0/2874212508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577d0c93c00 0x5577d0192680 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 >> 172.21.15.138:0/2874212508 conn(0x5577d0c93400 msgr2=0x5577cffb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 shutdown_connections 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/2874212508 wait complete. 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0e04000 msgr2=0x5577d0191080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0e04000 0x5577d0191080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x5577d0db4450 tx=0x5577cffc14a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 msgr2=0x5577d0190580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5577d0d6aea0 tx=0x5577cffc0c60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 shutdown_connections 2024-04-27T05:21:39.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5577d0e04000 0x5577d0191080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 --2- 172.21.15.138:0/225864909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577cfeaf000 0x5577d0190580 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 >> 172.21.15.138:0/225864909 conn(0x5577cfeaec00 msgr2=0x5577cffb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 shutdown_connections 2024-04-27T05:21:39.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.506+0000 7f3c846bb880 1 -- 172.21.15.138:0/225864909 wait complete. 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "num_shards": 11, 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "tenant": "", 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "zonegroup": "7c1a7efe-08a0-41e8-a0e4-e18cf496a33b", 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_rule": "default-placement", 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "explicit_placement": { 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "", 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "", 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "" 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:39.530 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "marker": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": "Normal", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "versioned": false, 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "versioning_enabled": false, 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "object_lock_enabled": false, 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_enabled": false, 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mtime": "2024-04-27T05:21:39.050974Z", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "creation_time": "2024-04-27T05:21:36.328858Z", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "usage": { 2024-04-27T05:21:39.531 INFO:teuthology.orchestra.run.smithi138.stdout: "rgw.main": { 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 0, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "size_actual": 0, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "size_utilized": 0, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb": 0, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb_actual": 0, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "size_kb_utilized": 0, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 0 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:39.532 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:39.533 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:39.533 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:39.533 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:39.533 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:21:39.050974Z', 'creation_time': '2024-04-27T05:21:36.328858Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T05:21:39.533 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-04-27T05:21:39.533 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-04-27T05:21:39.533 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-04-27T05:21:39.611 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.611 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.609+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.609+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f07f42880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f07f42880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d5d5186600 con 0x55d5d4eaf400 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f07f42880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d5d4eee280 con 0x55d5d4eaf000 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f04948640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f04147640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55d5d4eaf400 0x55d5d4fb6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42894/0 (socket says 172.21.15.138:42894) 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f04948640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59746/0 (socket says 172.21.15.138:59746) 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f04147640 1 -- 172.21.15.138:0/379205762 learned_addr learned my addr 172.21.15.138:0/379205762 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f04948640 1 -- 172.21.15.138:0/379205762 >> v1:172.21.15.114:6789/0 conn(0x55d5d4eaf400 legacy=0x55d5d4fb6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.610+0000 7f9f04948640 1 -- 172.21.15.138:0/379205762 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4eb3380 con 0x55d5d4eaf000 2024-04-27T05:21:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.612+0000 7f9f04948640 1 --2- 172.21.15.138:0/379205762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55d5d4fbad50 tx=0x55d5d4fc0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c25256e4516cde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.612+0000 7f9f03946640 1 -- 172.21.15.138:0/379205762 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec7ce0 con 0x55d5d4eaf000 2024-04-27T05:21:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.612+0000 7f9f03946640 1 -- 172.21.15.138:0/379205762 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5187b00 con 0x55d5d4eaf000 2024-04-27T05:21:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.612+0000 7f9f03946640 1 -- 172.21.15.138:0/379205762 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec7e40 con 0x55d5d4eaf000 2024-04-27T05:21:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 -- 172.21.15.138:0/379205762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 msgr2=0x55d5d5190580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 --2- 172.21.15.138:0/379205762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55d5d4fbad50 tx=0x55d5d4fc0de0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 -- 172.21.15.138:0/379205762 shutdown_connections 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 --2- 172.21.15.138:0/379205762 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 -- 172.21.15.138:0/379205762 >> 172.21.15.138:0/379205762 conn(0x55d5d4eaec00 msgr2=0x55d5d4fb6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 -- 172.21.15.138:0/379205762 shutdown_connections 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 -- 172.21.15.138:0/379205762 wait complete. 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.613+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f07f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f07f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d4eee780 con 0x55d5d4eaf000 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f04948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f04948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59758/0 (socket says 172.21.15.138:59758) 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f04948640 1 -- 172.21.15.138:0/2472513681 learned_addr learned my addr 172.21.15.138:0/2472513681 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f04948640 1 -- 172.21.15.138:0/2472513681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4eb2ea0 con 0x55d5d4eaf000 2024-04-27T05:21:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f04948640 1 --2- 172.21.15.138:0/2472513681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55d5d5cb3560 tx=0x55d5d4fc0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9c6952991d519d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f03145640 1 -- 172.21.15.138:0/2472513681 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec74a0 con 0x55d5d4eaf000 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f03145640 1 -- 172.21.15.138:0/2472513681 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5187e00 con 0x55d5d4eaf000 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.614+0000 7f9f03145640 1 -- 172.21.15.138:0/2472513681 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec7ce0 con 0x55d5d4eaf000 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- 172.21.15.138:0/2472513681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 msgr2=0x55d5d5190580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 --2- 172.21.15.138:0/2472513681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55d5d5cb3560 tx=0x55d5d4fc0cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- 172.21.15.138:0/2472513681 shutdown_connections 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 --2- 172.21.15.138:0/2472513681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- 172.21.15.138:0/2472513681 >> 172.21.15.138:0/2472513681 conn(0x55d5d4eaec00 msgr2=0x55d5d4fb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- 172.21.15.138:0/2472513681 shutdown_connections 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- 172.21.15.138:0/2472513681 wait complete. 2024-04-27T05:21:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f07f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d4eee8c0 con 0x55d5d4eaf000 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f04948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f04948640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59764/0 (socket says 172.21.15.138:59764) 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f04948640 1 -- 172.21.15.138:0/3070308535 learned_addr learned my addr 172.21.15.138:0/3070308535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.615+0000 7f9f04948640 1 -- 172.21.15.138:0/3070308535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4eb36c0 con 0x55d5d4eaf000 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f04948640 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55d5d5d6ade0 tx=0x55d5d4fc0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f01942640 1 -- 172.21.15.138:0/3070308535 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec7e40 con 0x55d5d4eaf000 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f01942640 1 -- 172.21.15.138:0/3070308535 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5c8e000 con 0x55d5d4eaf000 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5d4eb31e0 con 0x55d5d4eaf000 2024-04-27T05:21:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f01942640 1 -- 172.21.15.138:0/3070308535 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec74a0 con 0x55d5d4eaf000 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5d4f20ea0 con 0x55d5d4eaf000 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f07f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.616+0000 7f9f07f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d4eeec80 con 0x55d5d5ca6c00 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f01942640 1 -- 172.21.15.138:0/3070308535 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d5d4ed8b00 con 0x55d5d4eaf000 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f05149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f05149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59766/0 (socket says 172.21.15.138:59766) 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f05149640 1 -- 172.21.15.138:0/1154483338 learned_addr learned my addr 172.21.15.138:0/1154483338 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f01942640 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5ca7c00 0x55d5d5191080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f01942640 1 -- 172.21.15.138:0/3070308535 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55d5d4ec1a40 con 0x55d5d4eaf000 2024-04-27T05:21:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f04948640 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5ca7c00 0x55d5d5191080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f05149640 1 -- 172.21.15.138:0/1154483338 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4f20d00 con 0x55d5d5ca6c00 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f05149640 1 --2- 172.21.15.138:0/1154483338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d5d5daf0e0 tx=0x55d5d4fc1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c651c1c529f114b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9f04948640 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5ca7c00 0x55d5d5191080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55d5d5dae2d0 tx=0x55d5d4fc0cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9eff13d640 1 -- 172.21.15.138:0/1154483338 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec7e40 con 0x55d5d5ca6c00 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9eff13d640 1 -- 172.21.15.138:0/1154483338 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5c8e600 con 0x55d5d5ca6c00 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.617+0000 7f9eff13d640 1 -- 172.21.15.138:0/1154483338 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec74a0 con 0x55d5d5ca6c00 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- 172.21.15.138:0/1154483338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 msgr2=0x55d5d5190b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1154483338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d5d5daf0e0 tx=0x55d5d4fc1980 comp rx=0 tx=0).stop 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- 172.21.15.138:0/1154483338 shutdown_connections 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1154483338 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- 172.21.15.138:0/1154483338 >> 172.21.15.138:0/1154483338 conn(0x55d5d4eaf400 msgr2=0x55d5d4fb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- 172.21.15.138:0/1154483338 shutdown_connections 2024-04-27T05:21:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- 172.21.15.138:0/1154483338 wait complete. 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f07f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d4ed2280 con 0x55d5d5ca6c00 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f05149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f05149640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59776/0 (socket says 172.21.15.138:59776) 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.618+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 learned_addr learned my addr 172.21.15.138:0/144046342 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4f20b60 con 0x55d5d5ca6c00 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9f05149640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55d5d5cb35c0 tx=0x55d5d4fc19e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9efd93a640 1 -- 172.21.15.138:0/144046342 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec7e40 con 0x55d5d5ca6c00 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9efd93a640 1 -- 172.21.15.138:0/144046342 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5c8e780 con 0x55d5d5ca6c00 2024-04-27T05:21:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5d4f209c0 con 0x55d5d5ca6c00 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9efd93a640 1 -- 172.21.15.138:0/144046342 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d4ec74a0 con 0x55d5d5ca6c00 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.619+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5d4f20820 con 0x55d5d5ca6c00 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9efd93a640 1 -- 172.21.15.138:0/144046342 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d5d4ed9600 con 0x55d5d5ca6c00 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9efd93a640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d522c000 0x55d5d5191600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9efd93a640 1 -- 172.21.15.138:0/144046342 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55d5d4ec1c00 con 0x55d5d5ca6c00 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f04147640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d522c000 0x55d5d5191600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5ca7000 0x55d5d5191b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5ca7400 con 0x55d5d5ca7000 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f05149640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5ca7000 0x55d5d5191b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f04147640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d522c000 0x55d5d5191600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d5d5e1b260 tx=0x55d5d51f2120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.620+0000 7f9f05149640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5ca7000 0x55d5d5191b80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 realm 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5ca7800 con 0x55d5d5ca7000 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522d000 con 0x55d5d5ca7000 2024-04-27T05:21:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522d400 con 0x55d5d5ca7000 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d522d800 0x55d5d5192100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522dc00 con 0x55d5d522d800 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.621+0000 7f9f04948640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d522d800 0x55d5d5192100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f04948640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d522d800 0x55d5d5192100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f07f42880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f07f42880 20 searching for the correct realm 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522d000 con 0x55d5d522d800 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f04948640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d522d000 0x55d5d5192c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522dc00 con 0x55d5d522d000 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.622+0000 7f9f04147640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d522d000 0x55d5d5192c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.623+0000 7f9f04147640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d522d000 0x55d5d5192c00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.623+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.623+0000 7f9f04147640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d522dc00 0x55d5d5193180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.623+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.623+0000 7f9f05149640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d522dc00 0x55d5d5193180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.623+0000 7f9f05149640 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d522dc00 0x55d5d5193180 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.624+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d000 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.625+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d000 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.626+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.627+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d000 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.628+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d000 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.629+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d800 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522dc00 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d5ca7000 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522cc00 con 0x55d5d522d000 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522d400 con 0x55d5d5ca7000 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.630+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5ca7400 con 0x55d5d5ca7000 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5ca7800 con 0x55d5d522d800 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 zone default found 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 4 Realm: () 2024-04-27T05:21:39.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 zonegroup default 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254000 con 0x55d5d522dc00 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.631+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254400 con 0x55d5d522d800 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254800 con 0x55d5d522d800 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 20 started sync module instance, tier type = 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.632+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254c00 con 0x55d5d522d000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.633+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.633+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94376891549952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d522d000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.633+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255400 con 0x55d5d5ca7000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.634+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94376891549952] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.634+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.634+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94376891551104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255800 con 0x55d5d5ca7000 2024-04-27T05:21:39.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.634+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255c00 con 0x55d5d522d800 2024-04-27T05:21:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.635+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.635+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94376891552256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254400 con 0x55d5d522d800 2024-04-27T05:21:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.635+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254800 con 0x55d5d522d000 2024-04-27T05:21:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.635+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94376891551104] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.636+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.636+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94376891553408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255c00 con 0x55d5d522d000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.636+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94376891552256] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.636+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254000 con 0x55d5d5ca7000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.637+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94376891553408] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.638+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.638+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94376891554560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5ca7800 con 0x55d5d5ca7000 2024-04-27T05:21:39.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.639+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5ca7400 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.639+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94376891554560] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.639+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.639+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94376891555712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d522d400 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.640+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5e0d800 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.640+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94376891555712] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.641+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.641+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94376891556864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5e0d400 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.641+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254c00 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.641+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94376891556864] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94376891558016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d522dc00 2024-04-27T05:21:39.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=0 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=1 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=2 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=3 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=4 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=5 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=6 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94376891558016] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 20 add_watcher() i=7 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.642+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254800 con 0x55d5d5ca7000 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.643+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.643+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94376891559168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255400 con 0x55d5d5ca7000 2024-04-27T05:21:39.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94376891559168] v29'152 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9eeb916640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f07f42880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f07f42880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f07f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f07f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d4ed28c0 con 0x55d5d522cc00 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f04147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f04147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59778/0 (socket says 172.21.15.138:59778) 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.644+0000 7f9f04147640 1 -- 172.21.15.138:0/1889569441 learned_addr learned my addr 172.21.15.138:0/1889569441 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.645+0000 7f9f04147640 1 -- 172.21.15.138:0/1889569441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4f20680 con 0x55d5d522cc00 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.645+0000 7f9f04147640 1 --2- 172.21.15.138:0/1889569441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55d5d520e900 tx=0x55d5d51f2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230dfd04ca85180e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.645+0000 7f9eeb115640 1 -- 172.21.15.138:0/1889569441 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d5256580 con 0x55d5d522cc00 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.645+0000 7f9eeb115640 1 -- 172.21.15.138:0/1889569441 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5c8e000 con 0x55d5d522cc00 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.645+0000 7f9eeb115640 1 -- 172.21.15.138:0/1889569441 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d52566e0 con 0x55d5d522cc00 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- 172.21.15.138:0/1889569441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 msgr2=0x55d5d5192680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1889569441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55d5d520e900 tx=0x55d5d51f2180 comp rx=0 tx=0).stop 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- 172.21.15.138:0/1889569441 shutdown_connections 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1889569441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- 172.21.15.138:0/1889569441 >> 172.21.15.138:0/1889569441 conn(0x55d5d5255800 msgr2=0x55d5d4fb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- 172.21.15.138:0/1889569441 shutdown_connections 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- 172.21.15.138:0/1889569441 wait complete. 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 Processor -- start 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- start start 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f07f42880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d4ed2780 con 0x55d5d522cc00 2024-04-27T05:21:39.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f04147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f04147640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59790/0 (socket says 172.21.15.138:59790) 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 learned_addr learned my addr 172.21.15.138:0/1595713275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.646+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d4f204e0 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9f04147640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55d5d520f830 tx=0x55d5d51f2ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9ee9912640 1 -- 172.21.15.138:0/1595713275 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d5256840 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9ee9912640 1 -- 172.21.15.138:0/1595713275 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5d5c8ea80 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5d4eb3380 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9ee9912640 1 -- 172.21.15.138:0/1595713275 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d5d5256580 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5d4eb2ea0 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.647+0000 7f9ee9912640 1 -- 172.21.15.138:0/1595713275 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d5d4ed9b80 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9ee9912640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5e0d800 0x55d5d5193700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9ee9912640 1 -- 172.21.15.138:0/1595713275 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x55d5d4ec1dc0 con 0x55d5d522cc00 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f07f42880 5 note: GC not initialized 2024-04-27T05:21:39.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f05149640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5e0d800 0x55d5d5193700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9ee710d640 20 reqs_thread_entry: start 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f07f42880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f05149640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5e0d800 0x55d5d5193700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d5d5e1b6e0 tx=0x55d5d51f28a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f07f42880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9ee5109640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:09 2024 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9ee48e7640 20 reqs_thread_entry: start 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d5326000 0x55d5d5328000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5326400 con 0x55d5d5326000 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.648+0000 7f9f04948640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d5326000 0x55d5d5328000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.649+0000 7f9f04948640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d5326000 0x55d5d5328000 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.649+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(1 [pgnls start_epoch 29 out=254b] v29'51 uv43 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5326000 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.649+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5326400 con 0x55d5d5326000 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.649+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(2 [pgnls start_epoch 29 out=131b] v29'4 uv4 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5326000 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.649+0000 7f9f04948640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d5326400 0x55d5d5329600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.650+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254400 con 0x55d5d5326400 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.650+0000 7f9f04147640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d5326400 0x55d5d5329600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.650+0000 7f9f04147640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d5326400 0x55d5d5329600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.650+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 [pgnls start_epoch 29 out=295b] v29'35 uv27 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.650+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5254400 con 0x55d5d5326000 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(4 [pgnls start_epoch 29 out=172b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5326000 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f04948640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5254400 0x55d5d5329b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5334000 con 0x55d5d5254400 2024-04-27T05:21:39.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f05149640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5254400 0x55d5d5329b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f05149640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5254400 0x55d5d5329b80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(5 [pgnls start_epoch 29 out=335b] v29'152 uv29 ondisk = 1) v8 ==== 144+0+335 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5254400 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f05149640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d5334000 0x55d5d532a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.651+0000 7f9f04948640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d5334000 0x55d5d532a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.652+0000 7f9f04948640 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d5334000 0x55d5d532a100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.652+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=213b] v29'23 uv23 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.652+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.652+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(7 [pgnls start_epoch 29 out=254b] v29'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.652+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(8 [pgnls start_epoch 29 out=295b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(9 [pgnls start_epoch 29 out=213b] v29'16 uv16 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(10 [pgnls start_epoch 29 out=172b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=336b] v29'33 uv25 ondisk = 1) v8 ==== 144+0+336 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.653+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(12 [pgnls start_epoch 29 out=131b] v29'29 uv21 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(13 [pgnls start_epoch 29 out=254b] v29'12 uv12 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(14 [pgnls start_epoch 29 out=90b] v29'29 uv21 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5254400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(15 [pgnls start_epoch 29 out=213b] v29'26 uv17 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5254400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.654+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(16 [pgnls start_epoch 29 out=254b] v29'17 uv17 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5254400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(17 [pgnls start_epoch 29 out=254b] v29'20 uv20 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5254400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(18 [pgnls start_epoch 29 out=213b] v29'25 uv17 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(19 [pgnls start_epoch 29 out=172b] v29'43 uv35 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.655+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5254400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.656+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=295b] v29'51 uv47 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5254400 2024-04-27T05:21:39.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.656+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.656+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(21 [pgnls start_epoch 29 out=276b] v29'13 uv13 ondisk = 1) v8 ==== 144+0+276 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.656+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.656+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=172b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.656+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5254400 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.657+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(23 [pgnls start_epoch 29 out=295b] v29'25 uv17 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5254400 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.657+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.657+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(24 [pgnls start_epoch 29 out=172b] v29'34 uv18 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5326000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.657+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.657+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(25 [pgnls start_epoch 29 out=213b] v29'49 uv25 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.657+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(26 [pgnls start_epoch 29 out=172b] v29'20 uv11 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5326000 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(27 [pgnls start_epoch 29 out=172b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(28 [pgnls start_epoch 29 out=90b] v29'19 uv11 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(29 [pgnls start_epoch 29 out=90b] v29'43 uv19 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d5326400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.658+0000 7f9f04147640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5254400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.659+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(30 [pgnls start_epoch 29 out=131b] v29'12 uv12 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5254400 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.659+0000 7f9f05149640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5334000 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.659+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(31 [pgnls start_epoch 29 out=295b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5334000 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.659+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5255000 con 0x55d5d5326000 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.659+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(32 [pgnls start_epoch 29 out=213b] v29'26 uv17 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d5326000 2024-04-27T05:21:39.684 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000078", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000070", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000104", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000026", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000028", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000040", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000015", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000069", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000095", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000003", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000047", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000035", 2024-04-27T05:21:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000037", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000024", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000000", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000031", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "data_loggenerations_metadata", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000076", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000113", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000005", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000011", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000058", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000052", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000088", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000080", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000090", 2024-04-27T05:21:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000110", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000096", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000087", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000008", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000006", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000029", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000089", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000022", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000016", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000041", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000018", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000092", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000014", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000112", 2024-04-27T05:21:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000007", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000021", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000064", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000071", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000074", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000081", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000009", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000121", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000125", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000082", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000105", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000059", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000077", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000032", 2024-04-27T05:21:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000053", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000091", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000065", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000083", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000010", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000045", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000002", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000116", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000034", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000101", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000079", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000049", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000117", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000044", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000066", 2024-04-27T05:21:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000068", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000085", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000073", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000038", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000118", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000036", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000103", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000119", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000098", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000027", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000019", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000039", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000100", 2024-04-27T05:21:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000093", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000004", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000063", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000122", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000057", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000054", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000114", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "data_log.0", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000012", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000084", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000043", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000111", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000048", 2024-04-27T05:21:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000020", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000099", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000056", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000072", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000062", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000115", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000033", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000025", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000123", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000108", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000094", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000060", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000109", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000013", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000051", 2024-04-27T05:21:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000106", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000017", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000023", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000107", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000046", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000067", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000086", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000050", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000102", 2024-04-27T05:21:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000124", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000001", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000126", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000055", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000061", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000097", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000042", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000075", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000120", 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "obj_delete_at_hint.0000000030" 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-04-27T05:21:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.660+0000 7f9eeb916640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.660+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94376891559168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5326800 con 0x55d5d5ca7000 2024-04-27T05:21:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.661+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94376891559168] v29'153 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.661+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94376891549952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5326c00 con 0x55d5d522d000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.662+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94376891549952] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.662+0000 7f9f07f42880 20 remove_watcher() i=0 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.662+0000 7f9f07f42880 2 removed watcher, disabling cache 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.662+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94376891551104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5327000 con 0x55d5d5ca7000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.663+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94376891551104] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.663+0000 7f9f07f42880 20 remove_watcher() i=1 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.663+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94376891552256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5327400 con 0x55d5d522d800 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.664+0000 7f9f04948640 1 -- 172.21.15.138:0/144046342 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94376891552256] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242fc0 con 0x55d5d522d800 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.664+0000 7f9f07f42880 20 remove_watcher() i=2 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.664+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94376891553408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5327800 con 0x55d5d522d000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.665+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94376891553408] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5243200 con 0x55d5d522d000 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.665+0000 7f9f07f42880 20 remove_watcher() i=3 2024-04-27T05:21:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.665+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94376891554560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5327c00 con 0x55d5d5ca7000 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.666+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94376891554560] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d5ca7000 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.666+0000 7f9f07f42880 20 remove_watcher() i=4 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.666+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94376891555712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5335800 con 0x55d5d522dc00 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.667+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94376891555712] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.667+0000 7f9f07f42880 20 remove_watcher() i=5 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.667+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94376891556864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d5335c00 con 0x55d5d522dc00 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.668+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94376891556864] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.668+0000 7f9f07f42880 20 remove_watcher() i=6 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.668+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94376891558016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5d51d6000 con 0x55d5d522dc00 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f05149640 1 -- 172.21.15.138:0/144046342 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94376891558016] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d5d5242900 con 0x55d5d522dc00 2024-04-27T05:21:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 20 remove_watcher() i=7 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d522dc00 msgr2=0x55d5d5193180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d522dc00 0x55d5d5193180 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5ca7000 msgr2=0x55d5d5191b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5ca7000 0x55d5d5191b80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d522d000 msgr2=0x55d5d5192c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d522d000 0x55d5d5192c00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d522d800 msgr2=0x55d5d5192100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d522d800 0x55d5d5192100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d522c000 msgr2=0x55d5d5191600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d522c000 0x55d5d5191600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55d5d5e1b260 tx=0x55d5d51f2120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 msgr2=0x55d5d5190b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/144046342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d5ca6c00 0x55d5d5190b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55d5d5cb35c0 tx=0x55d5d4fc19e0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 reap_dead start 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f04147640 1 -- 172.21.15.138:0/144046342 reap_dead start 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 shutdown_connections 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 >> 172.21.15.138:0/144046342 conn(0x55d5d4eaf400 msgr2=0x55d5d4fb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 shutdown_connections 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/144046342 wait complete. 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d5326000 msgr2=0x55d5d5328000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d5d5326000 0x55d5d5328000 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5254400 msgr2=0x55d5d5329b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d5d5254400 0x55d5d5329b80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d5334000 msgr2=0x55d5d532a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d5d5334000 0x55d5d532a100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d5326400 msgr2=0x55d5d5329600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.669+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d5d5326400 0x55d5d5329600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5e0d800 msgr2=0x55d5d5193700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5e0d800 0x55d5d5193700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d5d5e1b6e0 tx=0x55d5d51f28a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 msgr2=0x55d5d5192680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55d5d520f830 tx=0x55d5d51f2ae0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f04948640 1 -- 172.21.15.138:0/1595713275 reap_dead start 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 shutdown_connections 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/1595713275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d522cc00 0x55d5d5192680 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 >> 172.21.15.138:0/1595713275 conn(0x55d5d5255800 msgr2=0x55d5d4fb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 shutdown_connections 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/1595713275 wait complete. 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5ca7c00 msgr2=0x55d5d5191080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5ca7c00 0x55d5d5191080 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55d5d5dae2d0 tx=0x55d5d4fc0cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 msgr2=0x55d5d5190580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55d5d5d6ade0 tx=0x55d5d4fc0c60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 shutdown_connections 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d5d5ca7c00 0x55d5d5191080 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 --2- 172.21.15.138:0/3070308535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d5d4eaf000 0x55d5d5190580 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 >> 172.21.15.138:0/3070308535 conn(0x55d5d4eaec00 msgr2=0x55d5d4fb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 shutdown_connections 2024-04-27T05:21:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.670+0000 7f9f07f42880 1 -- 172.21.15.138:0/3070308535 wait complete. 2024-04-27T05:21:39.706 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-04-27T05:21:39.706 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-27T05:21:39.707 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-27T05:21:39.707 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-27T05:21:39.784 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.784 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be620f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56127d159000 0x56127d43a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be620f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56127d430600 con 0x56127d159400 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be620f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x56127d198280 con 0x56127d159000 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be264d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56127d159000 0x56127d43a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be1e4c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x56127d159400 0x56127d260800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42908/0 (socket says 172.21.15.138:42908) 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1717287775 learned_addr learned my addr 172.21.15.138:0/1717287775 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be264d640 1 -- 172.21.15.138:0/1717287775 >> v1:172.21.15.114:6789/0 conn(0x56127d159400 legacy=0x56127d260800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be264d640 1 -- 172.21.15.138:0/1717287775 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d15d380 con 0x56127d159000 2024-04-27T05:21:39.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.783+0000 7f1be164b640 1 -- 172.21.15.138:0/1717287775 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657566595 0 0) 0x56127d15c9c0 con 0x56127d159400 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.785+0000 7f1be264d640 1 --2- 172.21.15.138:0/1717287775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x56127d264d50 tx=0x56127d26ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8a8f67056fcc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.785+0000 7f1be164b640 1 -- 172.21.15.138:0/1717287775 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d171ce0 con 0x56127d159000 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.785+0000 7f1be164b640 1 -- 172.21.15.138:0/1717287775 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127d431b00 con 0x56127d159000 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.785+0000 7f1be164b640 1 -- 172.21.15.138:0/1717287775 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d171e40 con 0x56127d159000 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- 172.21.15.138:0/1717287775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 msgr2=0x56127d43a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 --2- 172.21.15.138:0/1717287775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x56127d264d50 tx=0x56127d26ade0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- 172.21.15.138:0/1717287775 shutdown_connections 2024-04-27T05:21:39.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 --2- 172.21.15.138:0/1717287775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- 172.21.15.138:0/1717287775 >> 172.21.15.138:0/1717287775 conn(0x56127d158c00 msgr2=0x56127d260000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- 172.21.15.138:0/1717287775 shutdown_connections 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- 172.21.15.138:0/1717287775 wait complete. 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.786+0000 7f1be620f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127d198780 con 0x56127d159000 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be264d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be264d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59820/0 (socket says 172.21.15.138:59820) 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be264d640 1 -- 172.21.15.138:0/2188835784 learned_addr learned my addr 172.21.15.138:0/2188835784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be264d640 1 -- 172.21.15.138:0/2188835784 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d15cea0 con 0x56127d159000 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be264d640 1 --2- 172.21.15.138:0/2188835784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x56127df5f590 tx=0x56127d26acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b05fe2e20cf0b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be0e4a640 1 -- 172.21.15.138:0/2188835784 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d1714a0 con 0x56127d159000 2024-04-27T05:21:39.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be0e4a640 1 -- 172.21.15.138:0/2188835784 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127d431e00 con 0x56127d159000 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be0e4a640 1 -- 172.21.15.138:0/2188835784 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d171ce0 con 0x56127d159000 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be620f880 1 -- 172.21.15.138:0/2188835784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 msgr2=0x56127d43a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.787+0000 7f1be620f880 1 --2- 172.21.15.138:0/2188835784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x56127df5f590 tx=0x56127d26acc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 -- 172.21.15.138:0/2188835784 shutdown_connections 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 --2- 172.21.15.138:0/2188835784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 -- 172.21.15.138:0/2188835784 >> 172.21.15.138:0/2188835784 conn(0x56127d158c00 msgr2=0x56127d261000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 -- 172.21.15.138:0/2188835784 shutdown_connections 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 -- 172.21.15.138:0/2188835784 wait complete. 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be620f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127d1988c0 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be264d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be264d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59828/0 (socket says 172.21.15.138:59828) 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be264d640 1 -- 172.21.15.138:0/3150913263 learned_addr learned my addr 172.21.15.138:0/3150913263 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be264d640 1 -- 172.21.15.138:0/3150913263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d15d6c0 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.788+0000 7f1be264d640 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x56127e014e10 tx=0x56127d26ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1bdf647640 1 -- 172.21.15.138:0/3150913263 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d171e40 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1bdf647640 1 -- 172.21.15.138:0/3150913263 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127df38000 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56127d15d1e0 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1bdf647640 1 -- 172.21.15.138:0/3150913263 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d1714a0 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56127d1caea0 con 0x56127d159000 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1be620f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1be620f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127d17c280 con 0x56127df50c00 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.789+0000 7f1bdf647640 1 -- 172.21.15.138:0/3150913263 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56127d182b00 con 0x56127d159000 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be2e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be2e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59830/0 (socket says 172.21.15.138:59830) 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be2e4e640 1 -- 172.21.15.138:0/2117547564 learned_addr learned my addr 172.21.15.138:0/2117547564 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1bdf647640 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127df51c00 0x56127d43b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1bdf647640 1 -- 172.21.15.138:0/3150913263 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56127d16ba40 con 0x56127d159000 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be264d640 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127df51c00 0x56127d43b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be2e4e640 1 -- 172.21.15.138:0/2117547564 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d1cad00 con 0x56127df50c00 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/2117547564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x56127e0595c0 tx=0x56127d26b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7655fd9635dd146 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1bdce42640 1 -- 172.21.15.138:0/2117547564 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d171e40 con 0x56127df50c00 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be264d640 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127df51c00 0x56127d43b080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x56127e0582d0 tx=0x56127d26acc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1bdce42640 1 -- 172.21.15.138:0/2117547564 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127df38600 con 0x56127df50c00 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1bdce42640 1 -- 172.21.15.138:0/2117547564 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d1714a0 con 0x56127df50c00 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be620f880 1 -- 172.21.15.138:0/2117547564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 msgr2=0x56127d43ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be620f880 1 --2- 172.21.15.138:0/2117547564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x56127e0595c0 tx=0x56127d26b980 comp rx=0 tx=0).stop 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be620f880 1 -- 172.21.15.138:0/2117547564 shutdown_connections 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be620f880 1 --2- 172.21.15.138:0/2117547564 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.790+0000 7f1be620f880 1 -- 172.21.15.138:0/2117547564 >> 172.21.15.138:0/2117547564 conn(0x56127d159400 msgr2=0x56127d263000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be620f880 1 -- 172.21.15.138:0/2117547564 shutdown_connections 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be620f880 1 -- 172.21.15.138:0/2117547564 wait complete. 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be620f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be620f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127d17d180 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be2e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be2e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59844/0 (socket says 172.21.15.138:59844) 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 learned_addr learned my addr 172.21.15.138:0/1493819963 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d1cab60 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.791+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x56127e015410 tx=0x56127d26b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1bdb63f640 1 -- 172.21.15.138:0/1493819963 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d171e40 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1bdb63f640 1 -- 172.21.15.138:0/1493819963 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127df38780 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56127d1ca9c0 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1bdb63f640 1 -- 172.21.15.138:0/1493819963 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d1714a0 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56127d1ca820 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1bdb63f640 1 -- 172.21.15.138:0/1493819963 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56127d183600 con 0x56127df50c00 2024-04-27T05:21:39.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1bdb63f640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127d4d6000 0x56127d43b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.792+0000 7f1bdb63f640 1 -- 172.21.15.138:0/1493819963 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56127d16bc00 con 0x56127df50c00 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127d4d6000 0x56127d43b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56127df51000 0x56127d43bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51400 con 0x56127df51000 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56127df51000 0x56127d43bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127d4d6000 0x56127d43b600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x56127d265410 tx=0x56127d49e120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56127df51000 0x56127d43bb80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 20 realm 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51800 con 0x56127df51000 2024-04-27T05:21:39.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.793+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7000 con 0x56127df51000 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7400 con 0x56127df51000 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56127d4d7800 0x56127d43c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7c00 con 0x56127d4d7800 2024-04-27T05:21:39.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be264d640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56127d4d7800 0x56127d43c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be264d640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56127d4d7800 0x56127d43c100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.794+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be620f880 20 searching for the correct realm 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7000 con 0x56127d4d7800 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be264d640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56127d4d7000 0x56127d43cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7c00 con 0x56127d4d7000 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56127d4d7000 0x56127d43cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.795+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56127d4d7000 0x56127d43cc00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56127d4d7c00 0x56127d43d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56127d4d7c00 0x56127d43d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56127d4d7c00 0x56127d43d180 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.796+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7000 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.797+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7000 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.798+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.799+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.800+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7000 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.801+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7000 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7800 2024-04-27T05:21:39.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7c00 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127df51000 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.802+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d6c00 con 0x56127d4d7000 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7400 con 0x56127df51000 2024-04-27T05:21:39.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51400 con 0x56127df51000 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51800 con 0x56127d4d7800 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 zone default found 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 4 Realm: () 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 zonegroup default 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.803+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500000 con 0x56127d4d7c00 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500400 con 0x56127d4d7800 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500800 con 0x56127d4d7800 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 started sync module instance, tier type = 2024-04-27T05:21:39.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.804+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500c00 con 0x56127d4d7000 2024-04-27T05:21:39.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.805+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.805+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94637410953472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501000 con 0x56127d4d7000 2024-04-27T05:21:39.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.805+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501400 con 0x56127df51000 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.806+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94637410953472] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.806+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.807+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94637410954624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501800 con 0x56127df51000 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.807+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501c00 con 0x56127d4d7800 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.807+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94637410954624] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.808+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.808+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94637410955776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500400 con 0x56127d4d7800 2024-04-27T05:21:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.808+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500800 con 0x56127d4d7000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.809+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94637410955776] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.809+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.809+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94637410956928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501c00 con 0x56127d4d7000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.809+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500000 con 0x56127df51000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.810+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.810+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94637410958080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51800 con 0x56127df51000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.810+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51400 con 0x56127d4d7c00 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.811+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94637410958080] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.811+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94637410956928] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.811+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.811+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94637410959232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d4d7400 con 0x56127d4d7c00 2024-04-27T05:21:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.811+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127e0b7800 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.812+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94637410959232] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.812+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.812+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94637410960384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127e0b7400 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.812+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500c00 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94637410960384] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94637410961536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501000 con 0x56127d4d7c00 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=0 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=1 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=2 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=3 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=4 2024-04-27T05:21:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=5 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.813+0000 7f1be620f880 20 add_watcher() i=6 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.814+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94637410961536] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.814+0000 7f1be620f880 20 add_watcher() i=7 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.814+0000 7f1be620f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.814+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500800 con 0x56127df51000 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.815+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.815+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94637410962688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501400 con 0x56127df51000 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94637410962688] v29'154 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1bc961b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be620f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be620f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be620f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be620f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127d17c780 con 0x56127d4d6c00 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be1e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be1e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59850/0 (socket says 172.21.15.138:59850) 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.816+0000 7f1be1e4c640 1 -- 172.21.15.138:0/2431658176 learned_addr learned my addr 172.21.15.138:0/2431658176 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1be1e4c640 1 -- 172.21.15.138:0/2431658176 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d1ca680 con 0x56127d4d6c00 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/2431658176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x56127d4bb620 tx=0x56127d49e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fa9e0b83544a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1bc8e1a640 1 -- 172.21.15.138:0/2431658176 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d502580 con 0x56127d4d6c00 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1bc8e1a640 1 -- 172.21.15.138:0/2431658176 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127df38000 con 0x56127d4d6c00 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1bc8e1a640 1 -- 172.21.15.138:0/2431658176 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d5026e0 con 0x56127d4d6c00 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1be620f880 1 -- 172.21.15.138:0/2431658176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 msgr2=0x56127d43c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1be620f880 1 --2- 172.21.15.138:0/2431658176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x56127d4bb620 tx=0x56127d49e180 comp rx=0 tx=0).stop 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1be620f880 1 -- 172.21.15.138:0/2431658176 shutdown_connections 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.817+0000 7f1be620f880 1 --2- 172.21.15.138:0/2431658176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 -- 172.21.15.138:0/2431658176 >> 172.21.15.138:0/2431658176 conn(0x56127d501800 msgr2=0x56127d263800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 -- 172.21.15.138:0/2431658176 shutdown_connections 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 -- 172.21.15.138:0/2431658176 wait complete. 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 Processor -- start 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 -- start start 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be620f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127d17c500 con 0x56127d4d6c00 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be1e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be1e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59860/0 (socket says 172.21.15.138:59860) 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.818+0000 7f1be1e4c640 1 -- 172.21.15.138:0/2599715751 learned_addr learned my addr 172.21.15.138:0/2599715751 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1be1e4c640 1 -- 172.21.15.138:0/2599715751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127d1ca4e0 con 0x56127d4d6c00 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1be1e4c640 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x56127d4bb830 tx=0x56127d49eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1bc7617640 1 -- 172.21.15.138:0/2599715751 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d502840 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1bc7617640 1 -- 172.21.15.138:0/2599715751 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56127df38a80 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56127d15d380 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1bc7617640 1 -- 172.21.15.138:0/2599715751 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56127d502580 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.819+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56127d15cea0 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1bc7617640 1 -- 172.21.15.138:0/2599715751 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56127d183b80 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1bc7617640 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127e0b7800 0x56127d43d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1bc7617640 1 -- 172.21.15.138:0/2599715751 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x56127d16bdc0 con 0x56127d4d6c00 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127e0b7800 0x56127d43d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be620f880 5 note: GC not initialized 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1bc4e12640 20 reqs_thread_entry: start 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be620f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be2e4e640 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127e0b7800 0x56127d43d700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x56127d4bb170 tx=0x56127d49e8a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be620f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1bc2e0e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:10 2024 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1bc25ec640 20 reqs_thread_entry: start 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be620f880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.820+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0000 con 0x56127df51000 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0400 con 0x56127d4d7000 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 20 disabling bucket name=myfoo 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0800 con 0x56127d4d7000 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.821+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0c00 con 0x56127d4d7000 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv7 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.822+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d1400 con 0x56127d4d7000 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(66 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'8 uv8 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be620f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be620f880 10 moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be620f880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be620f880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94637410964992 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d1c00 con 0x56127d4d7800 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94637410955776 notify 124554051595 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x56127e05ed00 con 0x56127d4d7800 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1bdce42640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051595 cookie 94637410955776 notifier 4645 bl.length()=725 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(67 notify.2 [notify cookie 94637410964992 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1bdce42640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1bdce42640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.823+0000 7f1bdce42640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1bdce42640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127df51400 con 0x56127d4d7800 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94637410964992 notify 124554051595 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56127e05eea0 con 0x56127d4d7800 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be620f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_sources= new_sources= 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be620f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential sources added=[] removed=[] 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be620f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_dests= new_dests= 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be620f880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential dests added=[] removed=[] 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.824+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0800 con 0x56127df51000 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v29'12 uv12 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be620f880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be620f880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be620f880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94637410964992 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0c00 con 0x56127df51000 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94637410958080 notify 124554051600 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x56127d49bd40 con 0x56127df51000 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1bdce42640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051600 cookie 94637410958080 notifier 4645 bl.length()=533 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1bdce42640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(70 notify.4 [notify cookie 94637410964992 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1bdce42640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.825+0000 7f1bdce42640 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500000 con 0x56127df51000 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.826+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.826+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94637410964992 notify 124554051600 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56127d559a00 con 0x56127df51000 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 1, 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:39.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:39.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:39.852 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.826+0000 7f1bc961b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.826+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94637410962688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d1800 con 0x56127df51000 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.827+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94637410962688] v29'155 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.827+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94637410953472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500c00 con 0x56127d4d7000 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.828+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94637410953472] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.828+0000 7f1be620f880 20 remove_watcher() i=0 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.828+0000 7f1be620f880 2 removed watcher, disabling cache 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.828+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94637410954624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d501000 con 0x56127df51000 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.829+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94637410954624] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.829+0000 7f1be620f880 20 remove_watcher() i=1 2024-04-27T05:21:39.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.829+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94637410955776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d500000 con 0x56127d4d7800 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.831+0000 7f1be264d640 1 -- 172.21.15.138:0/1493819963 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94637410955776] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4eefc0 con 0x56127d4d7800 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.831+0000 7f1be620f880 20 remove_watcher() i=2 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.831+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94637410956928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0000 con 0x56127d4d7000 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.831+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94637410956928] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ef200 con 0x56127d4d7000 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.831+0000 7f1be620f880 20 remove_watcher() i=3 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.831+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94637410958080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0800 con 0x56127df51000 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.832+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94637410958080] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127df51000 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.832+0000 7f1be620f880 20 remove_watcher() i=4 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.832+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94637410959232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0400 con 0x56127d4d7c00 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.833+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94637410959232] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.833+0000 7f1be620f880 20 remove_watcher() i=5 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.833+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94637410960384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127d5d0c00 con 0x56127d4d7c00 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.835+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94637410960384] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.835+0000 7f1be620f880 20 remove_watcher() i=6 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.835+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94637410961536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56127e0ba000 con 0x56127d4d7c00 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be2e4e640 1 -- 172.21.15.138:0/1493819963 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94637410961536] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56127d4ee900 con 0x56127d4d7c00 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 20 remove_watcher() i=7 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56127d4d7c00 msgr2=0x56127d43d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56127d4d7c00 0x56127d43d180 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56127df51000 msgr2=0x56127d43bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56127df51000 0x56127d43bb80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56127d4d7000 msgr2=0x56127d43cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56127d4d7000 0x56127d43cc00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56127d4d7800 msgr2=0x56127d43c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.836+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56127d4d7800 0x56127d43c100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127d4d6000 msgr2=0x56127d43b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127d4d6000 0x56127d43b600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x56127d265410 tx=0x56127d49e120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 msgr2=0x56127d43ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 --2- 172.21.15.138:0/1493819963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127df50c00 0x56127d43ab00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x56127e015410 tx=0x56127d26b9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 reap_dead start 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be1e4c640 1 -- 172.21.15.138:0/1493819963 reap_dead start 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 shutdown_connections 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 >> 172.21.15.138:0/1493819963 conn(0x56127d159400 msgr2=0x56127d263000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 shutdown_connections 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/1493819963 wait complete. 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127e0b7800 msgr2=0x56127d43d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127e0b7800 0x56127d43d700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x56127d4bb170 tx=0x56127d49e8a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 msgr2=0x56127d43c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x56127d4bb830 tx=0x56127d49eae0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 shutdown_connections 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127e0b7800 0x56127d43d700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 --2- 172.21.15.138:0/2599715751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d4d6c00 0x56127d43c680 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 >> 172.21.15.138:0/2599715751 conn(0x56127d501800 msgr2=0x56127d263800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 shutdown_connections 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/2599715751 wait complete. 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.837+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127df51c00 msgr2=0x56127d43b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127df51c00 0x56127d43b080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x56127e0582d0 tx=0x56127d26acc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 msgr2=0x56127d43a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x56127e014e10 tx=0x56127d26ac60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 shutdown_connections 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56127df51c00 0x56127d43b080 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 --2- 172.21.15.138:0/3150913263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127d159000 0x56127d43a580 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 >> 172.21.15.138:0/3150913263 conn(0x56127d158c00 msgr2=0x56127d261000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 shutdown_connections 2024-04-27T05:21:39.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.838+0000 7f1be620f880 1 -- 172.21.15.138:0/3150913263 wait complete. 2024-04-27T05:21:39.863 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:39.865 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-27T05:21:39.865 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-27T05:21:39.865 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-27T05:21:39.907 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:39.908 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f31e13880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5604fc6f5400 0x5604fc9d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f31e13880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5604fc9cc600 con 0x5604fc6f5000 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f31e13880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5604fc734280 con 0x5604fc6f5400 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f2db71640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5604fc6f5400 0x5604fc9d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f2e372640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5604fc6f5000 0x5604fc7fc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42914/0 (socket says 172.21.15.138:42914) 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.906+0000 7f6f2e372640 1 -- 172.21.15.138:0/1334864824 learned_addr learned my addr 172.21.15.138:0/1334864824 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:39.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.907+0000 7f6f2db71640 1 -- 172.21.15.138:0/1334864824 >> v1:172.21.15.114:6789/0 conn(0x5604fc6f5000 legacy=0x5604fc7fc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.907+0000 7f6f2d370640 1 -- 172.21.15.138:0/1334864824 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977070295 0 0) 0x5604fc6f89c0 con 0x5604fc6f5000 2024-04-27T05:21:39.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.907+0000 7f6f2db71640 1 -- 172.21.15.138:0/1334864824 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc6f9380 con 0x5604fc6f5400 2024-04-27T05:21:39.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.908+0000 7f6f2db71640 1 --2- 172.21.15.138:0/1334864824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5400 0x5604fc9d6580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5604fc800c00 tx=0x5604fc806de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bb23efa9814cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.908+0000 7f6f2d370640 1 -- 172.21.15.138:0/1334864824 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70dce0 con 0x5604fc6f5400 2024-04-27T05:21:39.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f2d370640 1 -- 172.21.15.138:0/1334864824 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fc9cdb00 con 0x5604fc6f5400 2024-04-27T05:21:39.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f2d370640 1 -- 172.21.15.138:0/1334864824 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70de40 con 0x5604fc6f5400 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 -- 172.21.15.138:0/1334864824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5400 msgr2=0x5604fc9d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 --2- 172.21.15.138:0/1334864824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5400 0x5604fc9d6580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x5604fc800c00 tx=0x5604fc806de0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 -- 172.21.15.138:0/1334864824 shutdown_connections 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 --2- 172.21.15.138:0/1334864824 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5400 0x5604fc9d6580 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 -- 172.21.15.138:0/1334864824 >> 172.21.15.138:0/1334864824 conn(0x5604fc6f4c00 msgr2=0x5604fc7fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 -- 172.21.15.138:0/1334864824 shutdown_connections 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.909+0000 7f6f31e13880 1 -- 172.21.15.138:0/1334864824 wait complete. 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f31e13880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f31e13880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604fc734780 con 0x5604fc6f5000 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f2e372640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f2e372640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59878/0 (socket says 172.21.15.138:59878) 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f2e372640 1 -- 172.21.15.138:0/1963069947 learned_addr learned my addr 172.21.15.138:0/1963069947 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f2e372640 1 -- 172.21.15.138:0/1963069947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc6f8ea0 con 0x5604fc6f5000 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f2e372640 1 --2- 172.21.15.138:0/1963069947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5604fd5a4c90 tx=0x5604fc806cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9d96fe1dd45fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.910+0000 7f6f2cb6f640 1 -- 172.21.15.138:0/1963069947 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70d4a0 con 0x5604fc6f5000 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f2cb6f640 1 -- 172.21.15.138:0/1963069947 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fc9cde00 con 0x5604fc6f5000 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f2cb6f640 1 -- 172.21.15.138:0/1963069947 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70dce0 con 0x5604fc6f5000 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- 172.21.15.138:0/1963069947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 msgr2=0x5604fc9d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 --2- 172.21.15.138:0/1963069947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5604fd5a4c90 tx=0x5604fc806cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- 172.21.15.138:0/1963069947 shutdown_connections 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 --2- 172.21.15.138:0/1963069947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- 172.21.15.138:0/1963069947 >> 172.21.15.138:0/1963069947 conn(0x5604fc6f4c00 msgr2=0x5604fc7fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- 172.21.15.138:0/1963069947 shutdown_connections 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- 172.21.15.138:0/1963069947 wait complete. 2024-04-27T05:21:39.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.911+0000 7f6f31e13880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604fc7348c0 con 0x5604fc6f5000 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2e372640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2e372640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59890/0 (socket says 172.21.15.138:59890) 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2e372640 1 -- 172.21.15.138:0/2755460242 learned_addr learned my addr 172.21.15.138:0/2755460242 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2e372640 1 -- 172.21.15.138:0/2755460242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc6f96c0 con 0x5604fc6f5000 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2e372640 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5604fd5a5350 tx=0x5604fc806c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2b36c640 1 -- 172.21.15.138:0/2755460242 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70de40 con 0x5604fc6f5000 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2b36c640 1 -- 172.21.15.138:0/2755460242 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fd5b2000 con 0x5604fc6f5000 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5604fc6f91e0 con 0x5604fc6f5000 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f2b36c640 1 -- 172.21.15.138:0/2755460242 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70d4a0 con 0x5604fc6f5000 2024-04-27T05:21:39.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.912+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604fc766ea0 con 0x5604fc6f5000 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f31e13880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f31e13880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604fc718280 con 0x5604fd4e9800 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2b36c640 1 -- 172.21.15.138:0/2755460242 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5604fc71f600 con 0x5604fc6f5000 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2eb73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2eb73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59892/0 (socket says 172.21.15.138:59892) 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2eb73640 1 -- 172.21.15.138:0/3999321240 learned_addr learned my addr 172.21.15.138:0/3999321240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2b36c640 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd64a000 0x5604fc9d7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2b36c640 1 -- 172.21.15.138:0/2755460242 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5604fc707a40 con 0x5604fc6f5000 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2e372640 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd64a000 0x5604fc9d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2eb73640 1 -- 172.21.15.138:0/3999321240 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc766d00 con 0x5604fd4e9800 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/3999321240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5604fd5fb440 tx=0x5604fc807a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b247ffe2a2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.913+0000 7f6f2e372640 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd64a000 0x5604fc9d7080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5604fd5fa450 tx=0x5604fc8074a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f28b67640 1 -- 172.21.15.138:0/3999321240 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70de40 con 0x5604fd4e9800 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f28b67640 1 -- 172.21.15.138:0/3999321240 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fd5b2300 con 0x5604fd4e9800 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f28b67640 1 -- 172.21.15.138:0/3999321240 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70d4a0 con 0x5604fd4e9800 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- 172.21.15.138:0/3999321240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 msgr2=0x5604fc9d6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 --2- 172.21.15.138:0/3999321240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5604fd5fb440 tx=0x5604fc807a40 comp rx=0 tx=0).stop 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- 172.21.15.138:0/3999321240 shutdown_connections 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 --2- 172.21.15.138:0/3999321240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- 172.21.15.138:0/3999321240 >> 172.21.15.138:0/3999321240 conn(0x5604fc6f5400 msgr2=0x5604fc7ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- 172.21.15.138:0/3999321240 shutdown_connections 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- 172.21.15.138:0/3999321240 wait complete. 2024-04-27T05:21:39.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.914+0000 7f6f31e13880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604fc719180 con 0x5604fd4e9800 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f2eb73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f2eb73640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59894/0 (socket says 172.21.15.138:59894) 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 learned_addr learned my addr 172.21.15.138:0/95296615 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc766b60 con 0x5604fd4e9800 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5604fd5a5a10 tx=0x5604fc807aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f27364640 1 -- 172.21.15.138:0/95296615 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70de40 con 0x5604fd4e9800 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f27364640 1 -- 172.21.15.138:0/95296615 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fd5b2480 con 0x5604fd4e9800 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5604fc7669c0 con 0x5604fd4e9800 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f27364640 1 -- 172.21.15.138:0/95296615 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fc70d4a0 con 0x5604fd4e9800 2024-04-27T05:21:39.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.915+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604fc766820 con 0x5604fd4e9800 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f27364640 1 -- 172.21.15.138:0/95296615 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5604fc71f8c0 con 0x5604fd4e9800 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f27364640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fca7a000 0x5604fc9d7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f27364640 1 -- 172.21.15.138:0/95296615 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5604fc707c00 con 0x5604fd4e9800 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f2db71640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fca7a000 0x5604fc9d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5604fd4e9c00 0x5604fc9d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7a400 con 0x5604fd4e9c00 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5604fd4e9c00 0x5604fc9d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.916+0000 7f6f2db71640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fca7a000 0x5604fc9d7600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5604fc800c00 tx=0x5604fca42120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5604fd4e9c00 0x5604fc9d7b80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 realm 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7a800 con 0x5604fd4e9c00 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7ac00 con 0x5604fd4e9c00 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7b000 con 0x5604fd4e9c00 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5604fca7b400 0x5604fc9d8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.917+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7a400 con 0x5604fca7b400 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.918+0000 7f6f2e372640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5604fca7b400 0x5604fc9d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.918+0000 7f6f2e372640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5604fca7b400 0x5604fc9d8100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.918+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.918+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.918+0000 7f6f31e13880 20 searching for the correct realm 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.918+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7a800 con 0x5604fca7b400 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2e372640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5604fca7a800 0x5604fc9d8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7a400 con 0x5604fca7a800 2024-04-27T05:21:39.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2db71640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5604fca7a800 0x5604fc9d8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2db71640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5604fca7a800 0x5604fc9d8c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2db71640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5604fca7a400 0x5604fc9d9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.919+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5604fca7a400 0x5604fc9d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5604fca7a400 0x5604fc9d9180 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.920+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a800 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.921+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a800 2024-04-27T05:21:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.922+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.923+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.924+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a800 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.925+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a800 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7b400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fd4e9c00 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.926+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4e8800 con 0x5604fca7a800 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7ac00 con 0x5604fd4e9c00 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7b000 con 0x5604fd4e9c00 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7b800 con 0x5604fca7b400 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 zone default found 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 4 Realm: () 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 zonegroup default 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.927+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7bc00 con 0x5604fca7a400 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8000 con 0x5604fca7b400 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8400 con 0x5604fca7b400 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 started sync module instance, tier type = 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.928+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8800 con 0x5604fca7a800 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.930+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.930+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94579417999616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8c00 con 0x5604fca7a800 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.930+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d9000 con 0x5604fd4e9c00 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.930+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94579417999616] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.931+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.931+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94579418000768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d9400 con 0x5604fd4e9c00 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.931+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d9800 con 0x5604fca7b400 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.932+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94579418000768] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.932+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.932+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94579418001920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8000 con 0x5604fca7b400 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.932+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8400 con 0x5604fca7a800 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.933+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94579418001920] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.933+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.933+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94579418003072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d9800 con 0x5604fca7a800 2024-04-27T05:21:39.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.933+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7b800 con 0x5604fd4e9c00 2024-04-27T05:21:39.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.934+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94579418003072] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.934+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.934+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94579418004224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7ac00 con 0x5604fd4e9c00 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.934+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7b000 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.936+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94579418004224] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.936+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.936+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94579418005376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fca7bc00 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.936+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd64b800 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.937+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94579418005376] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.937+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.937+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94579418006528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8800 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.937+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8c00 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94579418007680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d8400 con 0x5604fca7a400 2024-04-27T05:21:39.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=0 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=1 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=2 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=3 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=4 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=5 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94579418006528] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.940+0000 7f6f31e13880 20 add_watcher() i=6 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.941+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94579418007680] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.941+0000 7f6f31e13880 20 add_watcher() i=7 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.941+0000 7f6f31e13880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.941+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d9800 con 0x5604fd4e9c00 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.941+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.941+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94579418008832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fd4d9000 con 0x5604fd4e9c00 2024-04-27T05:21:39.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.942+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94579418008832] v29'156 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.942+0000 7f6f15340640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f31e13880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f31e13880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f31e13880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f31e13880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604fc718780 con 0x5604fd4d9c00 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f2db71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f2db71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59898/0 (socket says 172.21.15.138:59898) 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f2db71640 1 -- 172.21.15.138:0/4182308384 learned_addr learned my addr 172.21.15.138:0/4182308384 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f2db71640 1 -- 172.21.15.138:0/4182308384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc766680 con 0x5604fd4d9c00 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.943+0000 7f6f2db71640 1 --2- 172.21.15.138:0/4182308384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5604fca5ac30 tx=0x5604fca420c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd1b8b6b025cc9c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f14b3f640 1 -- 172.21.15.138:0/4182308384 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fd4da580 con 0x5604fd4d9c00 2024-04-27T05:21:39.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f14b3f640 1 -- 172.21.15.138:0/4182308384 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fd5b2000 con 0x5604fd4d9c00 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f14b3f640 1 -- 172.21.15.138:0/4182308384 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fd4da6e0 con 0x5604fd4d9c00 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 -- 172.21.15.138:0/4182308384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 msgr2=0x5604fc9d8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 --2- 172.21.15.138:0/4182308384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5604fca5ac30 tx=0x5604fca420c0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 -- 172.21.15.138:0/4182308384 shutdown_connections 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 --2- 172.21.15.138:0/4182308384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 -- 172.21.15.138:0/4182308384 >> 172.21.15.138:0/4182308384 conn(0x5604fd4d9400 msgr2=0x5604fc7ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 -- 172.21.15.138:0/4182308384 shutdown_connections 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.944+0000 7f6f31e13880 1 -- 172.21.15.138:0/4182308384 wait complete. 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f31e13880 1 Processor -- start 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f31e13880 1 -- start start 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f31e13880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f31e13880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604fc718500 con 0x5604fd4d9c00 2024-04-27T05:21:39.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f2db71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f2db71640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:59910/0 (socket says 172.21.15.138:59910) 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f2db71640 1 -- 172.21.15.138:0/2075470313 learned_addr learned my addr 172.21.15.138:0/2075470313 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f2db71640 1 -- 172.21.15.138:0/2075470313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604fc6f8ea0 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.945+0000 7f6f2db71640 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x5604fca5b7a0 tx=0x5604fca42000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f1333c640 1 -- 172.21.15.138:0/2075470313 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fd4da840 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f1333c640 1 -- 172.21.15.138:0/2075470313 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604fd5b2780 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5604fc6f96c0 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f1333c640 1 -- 172.21.15.138:0/2075470313 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5604fd4da580 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604fc6f91e0 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f1333c640 1 -- 172.21.15.138:0/2075470313 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5604fc71eb00 con 0x5604fd4d9c00 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.946+0000 7f6f1333c640 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd4d8800 0x5604fc9d9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:39.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f1333c640 1 -- 172.21.15.138:0/2075470313 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x5604fc707dc0 con 0x5604fd4d9c00 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f31e13880 5 note: GC not initialized 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd4d8800 0x5604fc9d9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f10b37640 20 reqs_thread_entry: start 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f31e13880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f2eb73640 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd4d8800 0x5604fc9d9700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5604fd668a20 tx=0x5604fca42960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f31e13880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f0eb33640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:10 2024 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f0e311640 20 reqs_thread_entry: start 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f31e13880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.947+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6a000 con 0x5604fd4e9c00 2024-04-27T05:21:39.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv12 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6a400 con 0x5604fca7a800 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 20 enabling bucket name=myfoo 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6a800 con 0x5604fca7a800 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:39.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 : miss 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 20 rados->read ofs=0 len=0 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.948+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6ac00 con 0x5604fca7a800 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(65 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv8 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 2024-04-27T05:21:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.949+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 8.2 8:42d329da:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6b400 con 0x5604fca7a800 2024-04-27T05:21:39.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(66 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'9 uv9 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f31e13880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f31e13880 10 moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f31e13880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f31e13880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1, ofs0, ns] 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94579418011136 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6bc00 con 0x5604fca7b400 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94579418001920 notify 124554051596 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5604fd608ea0 con 0x5604fca7b400 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f28b67640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051596 cookie 94579418001920 notifier 4653 bl.length()=725 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(67 notify.2 [notify cookie 94579418011136 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f28b67640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 info.flags=0x17 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f28b67640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1 to cache LRU end 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.950+0000 7f6f28b67640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f28b67640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa2000 con 0x5604fca7b400 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== watch-notify(notify_complete (2) cookie 94579418011136 notify 124554051596 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5604fd609040 con 0x5604fca7b400 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f31e13880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_sources= new_sources= 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f31e13880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential sources added=[] removed=[] 2024-04-27T05:21:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f31e13880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): orig_dests= new_dests= 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f31e13880 20 handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.1]): potential dests added=[] removed=[] 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.951+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa2c00 con 0x5604fd4e9c00 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v29'13 uv13 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f31e13880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f31e13880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f31e13880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94579418011136 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa3000 con 0x5604fd4e9c00 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94579418004224 notify 124554051601 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5604fca41d40 con 0x5604fd4e9c00 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f28b67640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051601 cookie 94579418004224 notifier 4653 bl.length()=533 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f28b67640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f28b67640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(70 notify.4 [notify cookie 94579418011136 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.952+0000 7f6f28b67640 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa2400 con 0x5604fd4e9c00 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.953+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== watch-notify(notify_complete (2) cookie 94579418011136 notify 124554051601 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5604fcaf3860 con 0x5604fd4e9c00 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.953+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:39.975 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:39.976 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:39.977 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:39.978 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:39.978 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:39.978 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:39.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.953+0000 7f6f15340640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:39.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.953+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94579418008832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa3400 con 0x5604fd4e9c00 2024-04-27T05:21:39.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.954+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94579418008832] v29'157 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.954+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94579417999616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa3800 con 0x5604fca7a800 2024-04-27T05:21:39.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.955+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94579417999616] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.955+0000 7f6f31e13880 20 remove_watcher() i=0 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.955+0000 7f6f31e13880 2 removed watcher, disabling cache 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.955+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94579418000768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa3c00 con 0x5604fd4e9c00 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.956+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94579418000768] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.956+0000 7f6f31e13880 20 remove_watcher() i=1 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.956+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94579418001920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcaa3800 con 0x5604fca7b400 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.957+0000 7f6f2e372640 1 -- 172.21.15.138:0/95296615 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94579418001920] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96fc0 con 0x5604fca7b400 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.957+0000 7f6f31e13880 20 remove_watcher() i=2 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.957+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94579418003072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6b400 con 0x5604fca7a800 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.958+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94579418003072] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca97200 con 0x5604fca7a800 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.958+0000 7f6f31e13880 20 remove_watcher() i=3 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.958+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94579418004224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6b800 con 0x5604fd4e9c00 2024-04-27T05:21:39.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.959+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94579418004224] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fd4e9c00 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.959+0000 7f6f31e13880 20 remove_watcher() i=4 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.959+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94579418005376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6ac00 con 0x5604fca7a400 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.960+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94579418005376] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.960+0000 7f6f31e13880 20 remove_watcher() i=5 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.960+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94579418006528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6a800 con 0x5604fca7a400 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.961+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94579418006528] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.961+0000 7f6f31e13880 20 remove_watcher() i=6 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.961+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94579418007680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5604fcb6b000 con 0x5604fca7a400 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f2eb73640 1 -- 172.21.15.138:0/95296615 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94579418007680] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5604fca96900 con 0x5604fca7a400 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 20 remove_watcher() i=7 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5604fca7a400 msgr2=0x5604fc9d9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5604fca7a400 0x5604fc9d9180 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5604fd4e9c00 msgr2=0x5604fc9d7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5604fd4e9c00 0x5604fc9d7b80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5604fca7a800 msgr2=0x5604fc9d8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5604fca7a800 0x5604fc9d8c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5604fca7b400 msgr2=0x5604fc9d8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5604fca7b400 0x5604fc9d8100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fca7a000 msgr2=0x5604fc9d7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fca7a000 0x5604fc9d7600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5604fc800c00 tx=0x5604fca42120 comp rx=0 tx=0).stop 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 msgr2=0x5604fc9d6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 --2- 172.21.15.138:0/95296615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4e9800 0x5604fc9d6b00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5604fd5a5a10 tx=0x5604fc807aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 reap_dead start 2024-04-27T05:21:39.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 shutdown_connections 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f2db71640 1 -- 172.21.15.138:0/95296615 reap_dead start 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 >> 172.21.15.138:0/95296615 conn(0x5604fc6f5400 msgr2=0x5604fc7ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 shutdown_connections 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.962+0000 7f6f31e13880 1 -- 172.21.15.138:0/95296615 wait complete. 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd4d8800 msgr2=0x5604fc9d9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd4d8800 0x5604fc9d9700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5604fd668a20 tx=0x5604fca42960 comp rx=0 tx=0).stop 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 msgr2=0x5604fc9d8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x5604fca5b7a0 tx=0x5604fca42000 comp rx=0 tx=0).stop 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 shutdown_connections 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd4d8800 0x5604fc9d9700 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2075470313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fd4d9c00 0x5604fc9d8680 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 >> 172.21.15.138:0/2075470313 conn(0x5604fd4d9400 msgr2=0x5604fc7ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 shutdown_connections 2024-04-27T05:21:39.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2075470313 wait complete. 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd64a000 msgr2=0x5604fc9d7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd64a000 0x5604fc9d7080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5604fd5fa450 tx=0x5604fc8074a0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 msgr2=0x5604fc9d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5604fd5a5350 tx=0x5604fc806c60 comp rx=0 tx=0).stop 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 shutdown_connections 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5604fd64a000 0x5604fc9d7080 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 --2- 172.21.15.138:0/2755460242 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604fc6f5000 0x5604fc9d6580 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 >> 172.21.15.138:0/2755460242 conn(0x5604fc6f4c00 msgr2=0x5604fc7fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 shutdown_connections 2024-04-27T05:21:39.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:39.963+0000 7f6f31e13880 1 -- 172.21.15.138:0/2755460242 wait complete. 2024-04-27T05:21:39.988 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:39.995 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T05:21:40.226 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-04-27T05:21:40.233 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:55.234 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-27T05:21:55.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-27T05:21:55.235 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-27T05:21:55.284 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:55.284 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.283+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102972a880 1 -- start start 2024-04-27T05:21:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102972a880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102972a880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fb2b37a600 con 0x55fb2b0a3400 2024-04-27T05:21:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102972a880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fb2b0e2280 con 0x55fb2b0a3000 2024-04-27T05:21:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102604d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102584c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55fb2b0a3400 0x55fb2b1aa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:44994/0 (socket says 172.21.15.138:44994) 2024-04-27T05:21:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102584c640 1 -- 172.21.15.138:0/2373884014 learned_addr learned my addr 172.21.15.138:0/2373884014 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102604d640 1 -- 172.21.15.138:0/2373884014 >> v1:172.21.15.114:6789/0 conn(0x55fb2b0a3400 legacy=0x55fb2b1aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.284+0000 7f102604d640 1 -- 172.21.15.138:0/2373884014 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b0a7380 con 0x55fb2b0a3000 2024-04-27T05:21:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.286+0000 7f102604d640 1 --2- 172.21.15.138:0/2373884014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55fb2b1aed50 tx=0x55fb2b1b4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb481bf94b967b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.286+0000 7f102504b640 1 -- 172.21.15.138:0/2373884014 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bbce0 con 0x55fb2b0a3000 2024-04-27T05:21:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.286+0000 7f102504b640 1 -- 172.21.15.138:0/2373884014 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2b37bb00 con 0x55fb2b0a3000 2024-04-27T05:21:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.286+0000 7f102504b640 1 -- 172.21.15.138:0/2373884014 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bbe40 con 0x55fb2b0a3000 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 -- 172.21.15.138:0/2373884014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 msgr2=0x55fb2b384580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 --2- 172.21.15.138:0/2373884014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55fb2b1aed50 tx=0x55fb2b1b4de0 comp rx=0 tx=0).stop 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 -- 172.21.15.138:0/2373884014 shutdown_connections 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 --2- 172.21.15.138:0/2373884014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 -- 172.21.15.138:0/2373884014 >> 172.21.15.138:0/2373884014 conn(0x55fb2b0a2c00 msgr2=0x55fb2b1aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 -- 172.21.15.138:0/2373884014 shutdown_connections 2024-04-27T05:21:55.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.287+0000 7f102972a880 1 -- 172.21.15.138:0/2373884014 wait complete. 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102972a880 1 -- start start 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102972a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102972a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b0e2780 con 0x55fb2b0a3000 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102604d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102604d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44638/0 (socket says 172.21.15.138:44638) 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102604d640 1 -- 172.21.15.138:0/404060536 learned_addr learned my addr 172.21.15.138:0/404060536 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102604d640 1 -- 172.21.15.138:0/404060536 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b0a6ea0 con 0x55fb2b0a3000 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.288+0000 7f102604d640 1 --2- 172.21.15.138:0/404060536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55fb2bea7560 tx=0x55fb2b1b4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb606e548257ef51 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102484a640 1 -- 172.21.15.138:0/404060536 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bb4a0 con 0x55fb2b0a3000 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102484a640 1 -- 172.21.15.138:0/404060536 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2b37be00 con 0x55fb2b0a3000 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102484a640 1 -- 172.21.15.138:0/404060536 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bbce0 con 0x55fb2b0a3000 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 -- 172.21.15.138:0/404060536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 msgr2=0x55fb2b384580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 --2- 172.21.15.138:0/404060536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55fb2bea7560 tx=0x55fb2b1b4cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 -- 172.21.15.138:0/404060536 shutdown_connections 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 --2- 172.21.15.138:0/404060536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 -- 172.21.15.138:0/404060536 >> 172.21.15.138:0/404060536 conn(0x55fb2b0a2c00 msgr2=0x55fb2b1ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 -- 172.21.15.138:0/404060536 shutdown_connections 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 -- 172.21.15.138:0/404060536 wait complete. 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.289+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102972a880 1 -- start start 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102972a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102972a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b0e28c0 con 0x55fb2b0a3000 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102604d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102604d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44642/0 (socket says 172.21.15.138:44642) 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102604d640 1 -- 172.21.15.138:0/4154359956 learned_addr learned my addr 172.21.15.138:0/4154359956 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102604d640 1 -- 172.21.15.138:0/4154359956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b0a76c0 con 0x55fb2b0a3000 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f102604d640 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55fb2bf5ede0 tx=0x55fb2b1b4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.290+0000 7f1023047640 1 -- 172.21.15.138:0/4154359956 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bbe40 con 0x55fb2b0a3000 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f1023047640 1 -- 172.21.15.138:0/4154359956 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2be82000 con 0x55fb2b0a3000 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb2b0a71e0 con 0x55fb2b0a3000 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f1023047640 1 -- 172.21.15.138:0/4154359956 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bb4a0 con 0x55fb2b0a3000 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb2b114ea0 con 0x55fb2b0a3000 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f102972a880 1 -- start start 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f102972a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f102972a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b0e2c80 con 0x55fb2be9ac00 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.291+0000 7f1023047640 1 -- 172.21.15.138:0/4154359956 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fb2b0ccb00 con 0x55fb2b0a3000 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102684e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f1023047640 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2be9bc00 0x55fb2b385080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102684e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44652/0 (socket says 172.21.15.138:44652) 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102684e640 1 -- 172.21.15.138:0/2148042413 learned_addr learned my addr 172.21.15.138:0/2148042413 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f1023047640 1 -- 172.21.15.138:0/4154359956 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5938+0+0 (secure 0 0 0) 0x55fb2b0b5a40 con 0x55fb2b0a3000 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102604d640 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2be9bc00 0x55fb2b385080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102684e640 1 -- 172.21.15.138:0/2148042413 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b114d00 con 0x55fb2be9ac00 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102684e640 1 --2- 172.21.15.138:0/2148042413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55fb2bf5f6e0 tx=0x55fb2b1b5f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2abec2edf99eff server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f102604d640 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2be9bc00 0x55fb2b385080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fb2bfa22a0 tx=0x55fb2b1b5c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f1020842640 1 -- 172.21.15.138:0/2148042413 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bbe40 con 0x55fb2be9ac00 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f1020842640 1 -- 172.21.15.138:0/2148042413 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2be82600 con 0x55fb2be9ac00 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.292+0000 7f1020842640 1 -- 172.21.15.138:0/2148042413 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bb4a0 con 0x55fb2be9ac00 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- 172.21.15.138:0/2148042413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 msgr2=0x55fb2b384b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 --2- 172.21.15.138:0/2148042413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55fb2bf5f6e0 tx=0x55fb2b1b5f80 comp rx=0 tx=0).stop 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- 172.21.15.138:0/2148042413 shutdown_connections 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 --2- 172.21.15.138:0/2148042413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- 172.21.15.138:0/2148042413 >> 172.21.15.138:0/2148042413 conn(0x55fb2b0a3400 msgr2=0x55fb2b1ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- 172.21.15.138:0/2148042413 shutdown_connections 2024-04-27T05:21:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- 172.21.15.138:0/2148042413 wait complete. 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- start start 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.293+0000 7f102972a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b0c6280 con 0x55fb2be9ac00 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102684e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102684e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44658/0 (socket says 172.21.15.138:44658) 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 learned_addr learned my addr 172.21.15.138:0/112208142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b114b60 con 0x55fb2be9ac00 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102684e640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55fb2bf5f980 tx=0x55fb2b1b5980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f101f03f640 1 -- 172.21.15.138:0/112208142 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bbe40 con 0x55fb2be9ac00 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f101f03f640 1 -- 172.21.15.138:0/112208142 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2be82780 con 0x55fb2be9ac00 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb2b1149c0 con 0x55fb2be9ac00 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f101f03f640 1 -- 172.21.15.138:0/112208142 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b0bb4a0 con 0x55fb2be9ac00 2024-04-27T05:21:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.294+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb2b114820 con 0x55fb2be9ac00 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.295+0000 7f101f03f640 1 -- 172.21.15.138:0/112208142 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fb2b0cd600 con 0x55fb2be9ac00 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.295+0000 7f101f03f640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2b420000 0x55fb2b385600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.295+0000 7f101f03f640 1 -- 172.21.15.138:0/112208142 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5938+0+0 (secure 0 0 0) 0x55fb2b0b5c00 con 0x55fb2be9ac00 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.295+0000 7f102584c640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2b420000 0x55fb2b385600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.295+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.296+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2be9b000 0x55fb2b385b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.296+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30) v8 -- 0x55fb2be9b400 con 0x55fb2be9b000 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.296+0000 7f102684e640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2be9b000 0x55fb2b385b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.296+0000 7f102584c640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2b420000 0x55fb2b385600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55fb2c010a80 tx=0x55fb2c012f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.296+0000 7f102684e640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2be9b000 0x55fb2b385b80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.972+0000 7f101f03f640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_map(31..31 src has 1..31) v4 ==== 312+0+0 (crc 0 0 0) 0x55fb2b0b5dc0 con 0x55fb2be9b000 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.972+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 20 realm 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b800 con 0x55fb2be9b000 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.973+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421000 con 0x55fb2be9b000 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421400 con 0x55fb2be9b000 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b421800 0x55fb2b386100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421c00 con 0x55fb2b421800 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.974+0000 7f102604d640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b421800 0x55fb2b386100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102604d640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b421800 0x55fb2b386100 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102972a880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102972a880 20 searching for the correct realm 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421000 con 0x55fb2b421800 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102604d640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b421000 0x55fb2b386c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421c00 con 0x55fb2b421000 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.975+0000 7f102584c640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b421000 0x55fb2b386c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.976+0000 7f102584c640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b421000 0x55fb2b386c00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.976+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.976+0000 7f102584c640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b421c00 0x55fb2b387180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.976+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.976+0000 7f102684e640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b421c00 0x55fb2b387180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b421c00 0x55fb2b387180 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.977+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.978+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421000 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.979+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.980+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.981+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.981+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.981+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.981+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.981+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.981+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.982+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.982+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.982+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.982+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421000 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.982+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.982+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.983+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.983+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.983+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.983+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.983+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.983+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.984+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.984+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421000 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.984+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.984+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421800 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.984+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.984+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421c00 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2be9b000 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b420c00 con 0x55fb2b421000 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102972a880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102972a880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102972a880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102972a880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421400 con 0x55fb2be9b000 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.985+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b400 con 0x55fb2be9b000 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b800 con 0x55fb2b421800 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 zone default found 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 4 Realm: () 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 zonegroup default 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444000 con 0x55fb2b421c00 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.986+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2b421800 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444800 con 0x55fb2b421800 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 started sync module instance, tier type = 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.987+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2b421000 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.988+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.988+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94537250228480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2b421000 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.988+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445400 con 0x55fb2be9b000 2024-04-27T05:21:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.989+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94537250228480] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.990+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(46 notify.1 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.990+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94537250229632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445800 con 0x55fb2be9b000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.990+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445c00 con 0x55fb2b421800 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.991+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94537250229632] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.991+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.991+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94537250230784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2b421800 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.991+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444800 con 0x55fb2b421000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.992+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.992+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94537250230784] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.992+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94537250231936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445c00 con 0x55fb2b421000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.992+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444000 con 0x55fb2be9b000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.994+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(52 notify.4 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.994+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94537250231936] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.994+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94537250233088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b800 con 0x55fb2be9b000 2024-04-27T05:21:56.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.994+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b400 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.995+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.995+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94537250234240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b421400 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.995+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2c001800 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.995+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94537250233088] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.996+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94537250234240] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.996+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.996+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94537250235392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2c001400 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.996+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94537250235392] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94537250236544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2b421c00 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=0 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=1 2024-04-27T05:21:56.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=2 2024-04-27T05:21:56.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=3 2024-04-27T05:21:56.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=4 2024-04-27T05:21:56.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=5 2024-04-27T05:21:56.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.997+0000 7f102972a880 20 add_watcher() i=6 2024-04-27T05:21:56.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.998+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94537250236544] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.998+0000 7f102972a880 20 add_watcher() i=7 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.998+0000 7f102972a880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.998+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444800 con 0x55fb2be9b000 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.999+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:55.999+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94537250237696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445400 con 0x55fb2be9b000 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94537250237696] v31'158 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f100d01b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102972a880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102972a880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102972a880 1 -- start start 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102972a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.000+0000 7f102972a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b0c68c0 con 0x55fb2b420c00 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.001+0000 7f102584c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.001+0000 7f102584c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44672/0 (socket says 172.21.15.138:44672) 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.001+0000 7f102584c640 1 -- 172.21.15.138:0/3244395600 learned_addr learned my addr 172.21.15.138:0/3244395600 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.001+0000 7f102584c640 1 -- 172.21.15.138:0/3244395600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b114680 con 0x55fb2b420c00 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.001+0000 7f102584c640 1 --2- 172.21.15.138:0/3244395600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55fb2b402900 tx=0x55fb2c012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906d5d36b65234f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.001+0000 7f100c81a640 1 -- 172.21.15.138:0/3244395600 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b446580 con 0x55fb2b420c00 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f100c81a640 1 -- 172.21.15.138:0/3244395600 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2be82000 con 0x55fb2b420c00 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f100c81a640 1 -- 172.21.15.138:0/3244395600 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b4466e0 con 0x55fb2b420c00 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 -- 172.21.15.138:0/3244395600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 msgr2=0x55fb2b386680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 --2- 172.21.15.138:0/3244395600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55fb2b402900 tx=0x55fb2c012f00 comp rx=0 tx=0).stop 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 -- 172.21.15.138:0/3244395600 shutdown_connections 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 --2- 172.21.15.138:0/3244395600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 -- 172.21.15.138:0/3244395600 >> 172.21.15.138:0/3244395600 conn(0x55fb2b445800 msgr2=0x55fb2b1ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 -- 172.21.15.138:0/3244395600 shutdown_connections 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.002+0000 7f102972a880 1 -- 172.21.15.138:0/3244395600 wait complete. 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102972a880 1 Processor -- start 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102972a880 1 -- start start 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102972a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102972a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb2b0c6780 con 0x55fb2b420c00 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102584c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102584c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44680/0 (socket says 172.21.15.138:44680) 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 learned_addr learned my addr 172.21.15.138:0/2956893620 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.003+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb2b1144e0 con 0x55fb2b420c00 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.004+0000 7f102584c640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55fb2b403b30 tx=0x55fb2c013080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.004+0000 7f100b017640 1 -- 172.21.15.138:0/2956893620 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b446840 con 0x55fb2b420c00 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.004+0000 7f100b017640 1 -- 172.21.15.138:0/2956893620 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb2be82a80 con 0x55fb2b420c00 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.004+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fb2b0a7380 con 0x55fb2b420c00 2024-04-27T05:21:56.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.004+0000 7f100b017640 1 -- 172.21.15.138:0/2956893620 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fb2b446580 con 0x55fb2b420c00 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.004+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb2b0a6ea0 con 0x55fb2b420c00 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f100b017640 1 -- 172.21.15.138:0/2956893620 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fb2b0cdb80 con 0x55fb2b420c00 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f100b017640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2c001800 0x55fb2b387700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f100b017640 1 -- 172.21.15.138:0/2956893620 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55fb2b0b5c00 con 0x55fb2b420c00 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f102684e640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2c001800 0x55fb2b387700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2c001400 0x55fb2b4e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8000 con 0x55fb2c001400 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f102604d640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2c001400 0x55fb2b4e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f102684e640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2c001800 0x55fb2b387700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55fb2b4032f0 tx=0x55fb2c013e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.005+0000 7f102604d640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2c001400 0x55fb2b4e4000 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.006+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'52 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.006+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b4e8400 0x55fb2b4e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.006+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8800 con 0x55fb2b4e8400 2024-04-27T05:21:56.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.007+0000 7f102584c640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b4e8400 0x55fb2b4e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.007+0000 7f102584c640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b4e8400 0x55fb2b4e4580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.008+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'25 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.008+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b4e8c00 0x55fb2b4e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.008+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.008+0000 7f102684e640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b4e8c00 0x55fb2b4e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.009+0000 7f102684e640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b4e8c00 0x55fb2b4e4b00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.009+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.009+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9400 con 0x55fb2c001400 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.010+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'159 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.011+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b4e9800 0x55fb2b4e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.011+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.011+0000 7f102604d640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b4e9800 0x55fb2b4e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.011+0000 7f102604d640 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b4e9800 0x55fb2b4e5080 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.012+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'50 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.012+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.013+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'44 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.013+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514400 con 0x55fb2c001400 2024-04-27T05:21:56.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.014+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.014+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2b4e8400 2024-04-27T05:21:56.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.016+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'35 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.016+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.017+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.017+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.018+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.018+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e9800 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.019+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'51 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.019+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515800 con 0x55fb2b4e8400 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.020+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.020+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.021+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'24 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.021+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514400 con 0x55fb2b4e8400 2024-04-27T05:21:56.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.022+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'52 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.022+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514c00 con 0x55fb2c001400 2024-04-27T05:21:56.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.023+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'160 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.023+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.024+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'44 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.024+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8000 con 0x55fb2b4e9800 2024-04-27T05:21:56.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.025+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'52 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.025+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.026+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'30 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.026+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9c00 con 0x55fb2c001400 2024-04-27T05:21:56.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.027+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.027+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2c001400 2024-04-27T05:21:56.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.028+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'161 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.028+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.029+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'45 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.029+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515000 con 0x55fb2b4e9800 2024-04-27T05:21:56.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.030+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'34 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.030+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.031+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.031+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.032+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'36 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.032+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.033+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.033+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.034+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'45 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.034+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2b4e8400 2024-04-27T05:21:56.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.036+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.036+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.037+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'30 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.037+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8800 con 0x55fb2b4e8400 2024-04-27T05:21:56.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.038+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'36 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.038+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.039+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'46 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.039+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.040+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'53 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.040+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2b4e9800 2024-04-27T05:21:56.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.041+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.041+0000 7f1008812640 20 reqs_thread_entry: start 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.041+0000 7f102972a880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f102972a880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f100680e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:26 2024 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f1005fec640 20 reqs_thread_entry: start 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566000 con 0x55fb2c001400 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566400 con 0x55fb2c001400 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.042+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566800 con 0x55fb2c001400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567000 con 0x55fb2b4e8400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv24 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567400 con 0x55fb2b4e8400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(38 gc.1 [call out=1702b] v0'0 uv24 ondisk = 0) v8 ==== 148+0+1702 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.043+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566000 con 0x55fb2c001400 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.044+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2c001400 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv17 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2c001400 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9400 con 0x55fb2b4e9800 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8000 con 0x55fb2b4e9800 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.045+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.046+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2c001400 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514c00 con 0x55fb2c001400 2024-04-27T05:21:56.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515000 con 0x55fb2c001400 2024-04-27T05:21:56.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e8400 2024-04-27T05:21:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.047+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv14 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515800 con 0x55fb2b4e8400 2024-04-27T05:21:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv14 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv14 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.048+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.049+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.050+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2b4e8400 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567000 con 0x55fb2b4e8400 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567400 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.051+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566400 con 0x55fb2b4e9800 2024-04-27T05:21:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566800 con 0x55fb2b4e9800 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566000 con 0x55fb2b4e8400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567400 con 0x55fb2b4e8400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.052+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566400 con 0x55fb2b4e8400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2c001400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2c001400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2c001400 2024-04-27T05:21:56.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.053+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.054+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.054+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.054+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.054+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.054+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.054+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e9800 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2b4e9800 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.055+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566000 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567400 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.056+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566400 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514000 con 0x55fb2c001400 2024-04-27T05:21:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514400 con 0x55fb2c001400 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2c001400 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.057+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 29 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 30 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9400 con 0x55fb2b4e9800 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.058+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8000 con 0x55fb2b4e9800 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 24 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 25 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 31 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 32 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.059+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 33 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 34 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 35 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 36 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.060+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 37 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 38 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 39 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 40 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.061+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.062+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 41 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.062+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.062+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 42 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.062+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.062+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.062+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2be9b400 con 0x55fb2b4e8400 2024-04-27T05:21:56.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.063+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.063+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8800 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.063+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.063+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514c00 con 0x55fb2b4e8c00 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 43 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 44 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 45 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.064+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567800 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.065+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.065+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.065+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.065+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567000 con 0x55fb2b4e8c00 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.065+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 46 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.065+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515400 con 0x55fb2b4e8c00 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 47 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514800 con 0x55fb2b4e8c00 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102684e640 1 -- 172.21.15.138:0/2956893620 <== osd.3 v2:172.21.15.138:6808/29866 48 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b4e8c00 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b514c00 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.066+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566800 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.067+0000 7f102584c640 1 -- 172.21.15.138:0/2956893620 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b4e8400 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.067+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567c00 con 0x55fb2b4e9800 2024-04-27T05:21:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.067+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 26 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.067+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566000 con 0x55fb2b4e9800 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.067+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 27 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.067+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567400 con 0x55fb2b4e9800 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.068+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 <== osd.2 v2:172.21.15.138:6809/29865 28 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b4e9800 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.068+0000 7f100d01b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.068+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94537250237696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566400 con 0x55fb2be9b000 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.069+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94537250237696] v31'162 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.069+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94537250228480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b444400 con 0x55fb2b421000 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.070+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94537250228480] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.070+0000 7f102972a880 20 remove_watcher() i=0 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.070+0000 7f102972a880 2 removed watcher, disabling cache 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.070+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94537250229632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b445000 con 0x55fb2be9b000 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.072+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94537250229632] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.072+0000 7f102972a880 20 remove_watcher() i=1 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.072+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94537250230784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b567800 con 0x55fb2b421800 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.073+0000 7f102604d640 1 -- 172.21.15.138:0/112208142 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94537250230784] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43ad80 con 0x55fb2b421800 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.073+0000 7f102972a880 20 remove_watcher() i=2 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.073+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94537250231936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566c00 con 0x55fb2b421000 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.074+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94537250231936] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43afc0 con 0x55fb2b421000 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.074+0000 7f102972a880 20 remove_watcher() i=3 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.074+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94537250233088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b566800 con 0x55fb2be9b000 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.075+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94537250233088] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2be9b000 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.075+0000 7f102972a880 20 remove_watcher() i=4 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.075+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94537250234240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e8800 con 0x55fb2b421c00 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.076+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94537250234240] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.076+0000 7f102972a880 20 remove_watcher() i=5 2024-04-27T05:21:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.076+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94537250235392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b4e9000 con 0x55fb2b421c00 2024-04-27T05:21:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.077+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94537250235392] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.077+0000 7f102972a880 20 remove_watcher() i=6 2024-04-27T05:21:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.077+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94537250236544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fb2b515c00 con 0x55fb2b421c00 2024-04-27T05:21:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.078+0000 7f102684e640 1 -- 172.21.15.138:0/112208142 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94537250236544] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fb2b43a6c0 con 0x55fb2b421c00 2024-04-27T05:21:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.078+0000 7f102972a880 20 remove_watcher() i=7 2024-04-27T05:21:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.078+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b421c00 msgr2=0x55fb2b387180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.078+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b421c00 0x55fb2b387180 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2be9b000 msgr2=0x55fb2b385b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2be9b000 0x55fb2b385b80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b421000 msgr2=0x55fb2b386c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b421000 0x55fb2b386c00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b421800 msgr2=0x55fb2b386100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b421800 0x55fb2b386100 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2b420000 msgr2=0x55fb2b385600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2b420000 0x55fb2b385600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55fb2c010a80 tx=0x55fb2c012f60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 msgr2=0x55fb2b384b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/112208142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2be9ac00 0x55fb2b384b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x55fb2bf5f980 tx=0x55fb2b1b5980 comp rx=0 tx=0).stop 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 reap_dead start 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102584c640 1 -- 172.21.15.138:0/112208142 reap_dead start 2024-04-27T05:21:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 shutdown_connections 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 >> 172.21.15.138:0/112208142 conn(0x55fb2b0a3400 msgr2=0x55fb2b1ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 shutdown_connections 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/112208142 wait complete. 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b4e8400 msgr2=0x55fb2b4e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fb2b4e8400 0x55fb2b4e4580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2c001400 msgr2=0x55fb2b4e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fb2c001400 0x55fb2b4e4000 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b4e9800 msgr2=0x55fb2b4e5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fb2b4e9800 0x55fb2b4e5080 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b4e8c00 msgr2=0x55fb2b4e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fb2b4e8c00 0x55fb2b4e4b00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2c001800 msgr2=0x55fb2b387700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2c001800 0x55fb2b387700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55fb2b4032f0 tx=0x55fb2c013e60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102604d640 1 -- 172.21.15.138:0/2956893620 reap_dead start 2024-04-27T05:21:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 msgr2=0x55fb2b386680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55fb2b403b30 tx=0x55fb2c013080 comp rx=0 tx=0).stop 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 shutdown_connections 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 --2- 172.21.15.138:0/2956893620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b420c00 0x55fb2b386680 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 >> 172.21.15.138:0/2956893620 conn(0x55fb2b445800 msgr2=0x55fb2b1ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 shutdown_connections 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.079+0000 7f102972a880 1 -- 172.21.15.138:0/2956893620 wait complete. 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2be9bc00 msgr2=0x55fb2b385080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2be9bc00 0x55fb2b385080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55fb2bfa22a0 tx=0x55fb2b1b5c20 comp rx=0 tx=0).stop 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 msgr2=0x55fb2b384580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55fb2bf5ede0 tx=0x55fb2b1b4c60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 shutdown_connections 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fb2be9bc00 0x55fb2b385080 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 --2- 172.21.15.138:0/4154359956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fb2b0a3000 0x55fb2b384580 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 >> 172.21.15.138:0/4154359956 conn(0x55fb2b0a2c00 msgr2=0x55fb2b1ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 shutdown_connections 2024-04-27T05:21:56.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.080+0000 7f102972a880 1 -- 172.21.15.138:0/4154359956 wait complete. 2024-04-27T05:21:56.090 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "tag": "2f19b8ff-2608-4d86-8f1c-430623205476.4229.15701436620086009928\u0000", 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:21:55.232074+0000", 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "objs": [ 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_1", 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.091 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_2", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_3", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_4", 2024-04-27T05:21:56.092 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_5", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_6", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:56.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: "pool": "default.rgw.buckets.data", 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: "oid": "2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_7", 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "", 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: "instance": "" 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:56.094 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-04-27T05:21:56.095 DEBUG:tasks.util.rgw: json result: [{'tag': '2f19b8ff-2608-4d86-8f1c-430623205476.4229.15701436620086009928\x00', 'time': '2024-04-27T05:21:55.232074+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_7', 'key': '', 'instance': ''}]}] 2024-04-27T05:21:56.095 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-04-27T05:21:56.095 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-04-27T05:21:56.095 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-04-27T05:21:56.135 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:56.136 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80cbd2a880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80cbd2a880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fdd7e42600 con 0x55fdd7b6b400 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80cbd2a880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fdd7baa280 con 0x55fdd7b6b000 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80c864d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80c7e4c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55fdd7b6b400 0x55fdd7c72800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45010/0 (socket says 172.21.15.138:45010) 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80c864d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44688/0 (socket says 172.21.15.138:44688) 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3389475424 learned_addr learned my addr 172.21.15.138:0/3389475424 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80c864d640 1 -- 172.21.15.138:0/3389475424 >> v1:172.21.15.114:6789/0 conn(0x55fdd7b6b400 legacy=0x55fdd7c72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.135+0000 7f80c864d640 1 -- 172.21.15.138:0/3389475424 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7b6f380 con 0x55fdd7b6b000 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.137+0000 7f80c864d640 1 --2- 172.21.15.138:0/3389475424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55fdd7c76d50 tx=0x55fdd7c7cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dcda9b0bb7eca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.137+0000 7f80c764b640 1 -- 172.21.15.138:0/3389475424 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b83ce0 con 0x55fdd7b6b000 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.137+0000 7f80c764b640 1 -- 172.21.15.138:0/3389475424 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd7e43b00 con 0x55fdd7b6b000 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.137+0000 7f80c764b640 1 -- 172.21.15.138:0/3389475424 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b83e40 con 0x55fdd7b6b000 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3389475424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 msgr2=0x55fdd7e4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3389475424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55fdd7c76d50 tx=0x55fdd7c7cde0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3389475424 shutdown_connections 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3389475424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3389475424 >> 172.21.15.138:0/3389475424 conn(0x55fdd7b6ac00 msgr2=0x55fdd7c72000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3389475424 shutdown_connections 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3389475424 wait complete. 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.138+0000 7f80cbd2a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd7baa780 con 0x55fdd7b6b000 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c864d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c864d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44698/0 (socket says 172.21.15.138:44698) 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c864d640 1 -- 172.21.15.138:0/1051380145 learned_addr learned my addr 172.21.15.138:0/1051380145 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c864d640 1 -- 172.21.15.138:0/1051380145 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7b6eea0 con 0x55fdd7b6b000 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c864d640 1 --2- 172.21.15.138:0/1051380145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55fdd896f560 tx=0x55fdd7c7ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d12c301213692d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c6e4a640 1 -- 172.21.15.138:0/1051380145 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b834a0 con 0x55fdd7b6b000 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c6e4a640 1 -- 172.21.15.138:0/1051380145 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd7e43e00 con 0x55fdd7b6b000 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.139+0000 7f80c6e4a640 1 -- 172.21.15.138:0/1051380145 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b83ce0 con 0x55fdd7b6b000 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- 172.21.15.138:0/1051380145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 msgr2=0x55fdd7e4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/1051380145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55fdd896f560 tx=0x55fdd7c7ccc0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- 172.21.15.138:0/1051380145 shutdown_connections 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/1051380145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- 172.21.15.138:0/1051380145 >> 172.21.15.138:0/1051380145 conn(0x55fdd7b6ac00 msgr2=0x55fdd7c73000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- 172.21.15.138:0/1051380145 shutdown_connections 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- 172.21.15.138:0/1051380145 wait complete. 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80cbd2a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd7baa8c0 con 0x55fdd7b6b000 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80c864d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80c864d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44704/0 (socket says 172.21.15.138:44704) 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80c864d640 1 -- 172.21.15.138:0/3508368191 learned_addr learned my addr 172.21.15.138:0/3508368191 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.140+0000 7f80c864d640 1 -- 172.21.15.138:0/3508368191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7b6f6c0 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80c864d640 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a26de0 tx=0x55fdd7c7cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80c5647640 1 -- 172.21.15.138:0/3508368191 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b83e40 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80c5647640 1 -- 172.21.15.138:0/3508368191 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd894a000 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80c5647640 1 -- 172.21.15.138:0/3508368191 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b834a0 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fdd7b6f1e0 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdd7bdcea0 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80cbd2a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.141+0000 7f80cbd2a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd7b8e280 con 0x55fdd8962c00 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c5647640 1 -- 172.21.15.138:0/3508368191 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fdd7b94dc0 con 0x55fdd7b6b000 2024-04-27T05:21:56.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c8e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c8e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44712/0 (socket says 172.21.15.138:44712) 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c5647640 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8963c00 0x55fdd7e4d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2158824599 learned_addr learned my addr 172.21.15.138:0/2158824599 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c5647640 1 -- 172.21.15.138:0/3508368191 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55fdd7b7da40 con 0x55fdd7b6b000 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c864d640 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8963c00 0x55fdd7e4d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2158824599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7bdcd00 con 0x55fdd8962c00 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c864d640 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8963c00 0x55fdd7e4d080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a6a2d0 tx=0x55fdd7c7dc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/2158824599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a6b590 tx=0x55fdd7c7df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f4006b36c23864 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c2e42640 1 -- 172.21.15.138:0/2158824599 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b83e40 con 0x55fdd8962c00 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c2e42640 1 -- 172.21.15.138:0/2158824599 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd894a600 con 0x55fdd8962c00 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.142+0000 7f80c2e42640 1 -- 172.21.15.138:0/2158824599 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b834a0 con 0x55fdd8962c00 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2158824599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 msgr2=0x55fdd7e4cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2158824599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a6b590 tx=0x55fdd7c7df80 comp rx=0 tx=0).stop 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2158824599 shutdown_connections 2024-04-27T05:21:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2158824599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2158824599 >> 172.21.15.138:0/2158824599 conn(0x55fdd7b6b400 msgr2=0x55fdd7c75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2158824599 shutdown_connections 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2158824599 wait complete. 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80cbd2a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd7b8f180 con 0x55fdd8962c00 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80c8e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80c8e4e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44714/0 (socket says 172.21.15.138:44714) 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.143+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 learned_addr learned my addr 172.21.15.138:0/2483192203 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7bdcb60 con 0x55fdd8962c00 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a26e70 tx=0x55fdd7c7da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80c163f640 1 -- 172.21.15.138:0/2483192203 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b83e40 con 0x55fdd8962c00 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80c163f640 1 -- 172.21.15.138:0/2483192203 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd894a780 con 0x55fdd8962c00 2024-04-27T05:21:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fdd7bdc9c0 con 0x55fdd8962c00 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80c163f640 1 -- 172.21.15.138:0/2483192203 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7b834a0 con 0x55fdd8962c00 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.144+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdd7bdc820 con 0x55fdd8962c00 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c163f640 1 -- 172.21.15.138:0/2483192203 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fdd7b94b00 con 0x55fdd8962c00 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c163f640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd7eee000 0x55fdd7e4d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c163f640 1 -- 172.21.15.138:0/2483192203 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55fdd7b7dc00 con 0x55fdd8962c00 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd7eee000 0x55fdd7e4d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8963000 0x55fdd7e4db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd8963000 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8963000 0x55fdd7e4db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd7eee000 0x55fdd7e4d600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55fdd8ad9260 tx=0x55fdd8adaf60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.145+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8963000 0x55fdd7e4db80 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 realm 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963800 con 0x55fdd8963000 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eef000 con 0x55fdd8963000 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eef400 con 0x55fdd8963000 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7eef800 0x55fdd7e4e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eefc00 con 0x55fdd7eef800 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.146+0000 7f80c864d640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7eef800 0x55fdd7e4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80c864d640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7eef800 0x55fdd7e4e100 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80cbd2a880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80cbd2a880 20 searching for the correct realm 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eef000 con 0x55fdd7eef800 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80c864d640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7eef000 0x55fdd7e4ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eefc00 con 0x55fdd7eef000 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.147+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7eef000 0x55fdd7e4ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.148+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7eef000 0x55fdd7e4ec00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.148+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.148+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7eefc00 0x55fdd7e4f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.148+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.148+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7eefc00 0x55fdd7e4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.148+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7eefc00 0x55fdd7e4f180 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.149+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef000 2024-04-27T05:21:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.150+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef000 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.151+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.152+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef000 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.153+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef000 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.154+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef800 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd8963000 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eeec00 con 0x55fdd7eef000 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80cbd2a880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80cbd2a880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80cbd2a880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80cbd2a880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.155+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eef400 con 0x55fdd8963000 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd8963000 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963800 con 0x55fdd7eef800 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 zone default found 2024-04-27T05:21:56.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 4 Realm: () 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 zonegroup default 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.156+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e000 con 0x55fdd7eefc00 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7eef800 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e800 con 0x55fdd7eef800 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.157+0000 7f80cbd2a880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.158+0000 7f80cbd2a880 20 started sync module instance, tier type = 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.158+0000 7f80cbd2a880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.158+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd7eef000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.159+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.159+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94548737149184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd7eef000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.159+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f400 con 0x55fdd8963000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.160+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94548737149184] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.160+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.160+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94548737150336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f800 con 0x55fdd8963000 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.160+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0fc00 con 0x55fdd7eef800 2024-04-27T05:21:56.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.161+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94548737150336] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.161+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.161+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94548737151488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7eef800 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.161+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e800 con 0x55fdd7eef000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.162+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.162+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94548737151488] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.162+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94548737152640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0fc00 con 0x55fdd7eef000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.162+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e000 con 0x55fdd8963000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.163+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.163+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94548737153792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963800 con 0x55fdd8963000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.163+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd7eefc00 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.163+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94548737152640] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.164+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.164+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94548737153792] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.164+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94548737154944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7eef400 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.164+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8ac9800 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.165+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94548737154944] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.165+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.165+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94548737156096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8ac9400 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.165+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94548737156096] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94548737157248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd7eefc00 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=0 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=1 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=2 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=3 2024-04-27T05:21:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=4 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=5 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.166+0000 7f80cbd2a880 20 add_watcher() i=6 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.167+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94548737157248] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7eefc00 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.167+0000 7f80cbd2a880 20 add_watcher() i=7 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.167+0000 7f80cbd2a880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.167+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e800 con 0x55fdd8963000 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.167+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.167+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94548737158400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f400 con 0x55fdd8963000 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.168+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94548737158400] v31'163 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd8963000 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.168+0000 7f80af61b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80cbd2a880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80cbd2a880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80cbd2a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80cbd2a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd7b8e780 con 0x55fdd7eeec00 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80c7e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80c7e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44718/0 (socket says 172.21.15.138:44718) 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80c7e4c640 1 -- 172.21.15.138:0/662178326 learned_addr learned my addr 172.21.15.138:0/662178326 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.169+0000 7f80c7e4c640 1 -- 172.21.15.138:0/662178326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7bdc680 con 0x55fdd7eeec00 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/662178326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x55fdd7eca900 tx=0x55fdd8adaf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a5078b0c90a2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80aee1a640 1 -- 172.21.15.138:0/662178326 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7f10580 con 0x55fdd7eeec00 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80aee1a640 1 -- 172.21.15.138:0/662178326 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd894a000 con 0x55fdd7eeec00 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80aee1a640 1 -- 172.21.15.138:0/662178326 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7f106e0 con 0x55fdd7eeec00 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 -- 172.21.15.138:0/662178326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 msgr2=0x55fdd7e4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/662178326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x55fdd7eca900 tx=0x55fdd8adaf00 comp rx=0 tx=0).stop 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 -- 172.21.15.138:0/662178326 shutdown_connections 2024-04-27T05:21:56.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/662178326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 -- 172.21.15.138:0/662178326 >> 172.21.15.138:0/662178326 conn(0x55fdd7f0f800 msgr2=0x55fdd7c75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 -- 172.21.15.138:0/662178326 shutdown_connections 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.170+0000 7f80cbd2a880 1 -- 172.21.15.138:0/662178326 wait complete. 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80cbd2a880 1 Processor -- start 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80cbd2a880 1 -- start start 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80cbd2a880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80cbd2a880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdd7b8e500 con 0x55fdd7eeec00 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80c7e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80c7e4c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44722/0 (socket says 172.21.15.138:44722) 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 learned_addr learned my addr 172.21.15.138:0/3986990452 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdd7bdc4e0 con 0x55fdd7eeec00 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.171+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55fdd7ecb830 tx=0x55fdd8adb080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80ad617640 1 -- 172.21.15.138:0/3986990452 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7f10840 con 0x55fdd7eeec00 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80ad617640 1 -- 172.21.15.138:0/3986990452 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdd894aa80 con 0x55fdd7eeec00 2024-04-27T05:21:56.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fdd7b6f380 con 0x55fdd7eeec00 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80ad617640 1 -- 172.21.15.138:0/3986990452 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55fdd7f10580 con 0x55fdd7eeec00 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdd7b6eea0 con 0x55fdd7eeec00 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80ad617640 1 -- 172.21.15.138:0/3986990452 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55fdd7b958c0 con 0x55fdd7eeec00 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.172+0000 7f80ad617640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8ac9800 0x55fdd7e4f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80ad617640 1 -- 172.21.15.138:0/3986990452 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55fdd7b7ddc0 con 0x55fdd7eeec00 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8ac9800 0x55fdd7e4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8ac9400 0x55fdd7fac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd8ac9400 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80c864d640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8ac9400 0x55fdd7fac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8ac9800 0x55fdd7e4f700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55fdd7ecb200 tx=0x55fdd8adbe60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.173+0000 7f80c864d640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8ac9400 0x55fdd7fac000 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.174+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'53 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.174+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7fb0400 0x55fdd7fac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.174+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd7fb0400 2024-04-27T05:21:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.174+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7fb0400 0x55fdd7fac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.175+0000 7f80c7e4c640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7fb0400 0x55fdd7fac580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.175+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'26 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.175+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7fb0c00 0x55fdd7facb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.175+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.176+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7fb0c00 0x55fdd7facb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.176+0000 7f80c8e4e640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7fb0c00 0x55fdd7facb00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.177+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.177+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd8ac9400 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.178+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'164 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.178+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7fb1800 0x55fdd7fad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.178+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.178+0000 7f80c864d640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7fb1800 0x55fdd7fad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.178+0000 7f80c864d640 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7fb1800 0x55fdd7fad080 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.179+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'53 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.179+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.180+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'46 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.180+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd8ac9400 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.181+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.181+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc800 con 0x55fdd7fb0400 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.182+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'37 uv14 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.182+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb1800 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.183+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.183+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.183+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.183+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd7fb1800 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.184+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'54 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.184+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb0400 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.185+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.185+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fddc00 con 0x55fdd7fb1800 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.186+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'25 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.186+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0400 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.187+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'54 uv26 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.187+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd8ac9400 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.188+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'165 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.188+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.189+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'47 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.189+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb1800 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.190+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'55 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.190+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.191+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'31 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.191+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd8ac9400 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.192+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.192+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd8ac9400 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.193+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'166 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.193+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.194+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'48 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.194+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb1800 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.195+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'35 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.195+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.196+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.196+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.197+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'37 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.197+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.198+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.198+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.199+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'47 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.199+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd7fb0400 2024-04-27T05:21:56.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.200+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.200+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.201+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'31 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.201+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd7fb0400 2024-04-27T05:21:56.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.202+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'38 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.202+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.203+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'49 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.203+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd7fb0400 2024-04-27T05:21:56.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.204+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'55 uv16 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.204+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd7fb1800 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80aae12640 20 reqs_thread_entry: start 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80cbd2a880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80cbd2a880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80a8e0e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:26 2024 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80a85ec640 20 reqs_thread_entry: start 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-04-27T05:21:56.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.205+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:33 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd8ac9400 2024-04-27T05:21:56.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.206+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(33 gc.3 [call] v31'167 uv167 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.206+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:34 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.206+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(34 gc.3 [call out=15b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.206+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.206+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:35 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(35 gc.3 [call out=24b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(36 gc.3 [call out=15b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:37 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(37 gc.3 [call out=15b] v0'0 uv167 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.207+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd8ac9400 2024-04-27T05:21:56.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.208+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(38 gc.3 [call] v31'168 uv168 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.208+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-04-27T05:21:56.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.208+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:39 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb1800 2024-04-27T05:21:56.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(39 gc.4 [call] v31'56 uv56 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:40 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(40 gc.4 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:41 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(41 gc.4 [call out=24b] v0'0 uv56 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.209+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:42 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.210+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(42 gc.4 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.210+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.210+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:43 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.210+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(43 gc.4 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.210+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.210+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fddc00 con 0x55fdd7fb1800 2024-04-27T05:21:56.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.211+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(44 gc.4 [call] v31'57 uv57 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.211+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.211+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:45 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.211+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(45 gc.5 [call] v31'48 uv48 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.211+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:46 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(46 gc.5 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:47 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(47 gc.5 [call out=24b] v0'0 uv48 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(49 gc.5 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.212+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.213+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(50 gc.5 [call] v31'49 uv49 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.213+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-04-27T05:21:56.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.213+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd8ac9400 2024-04-27T05:21:56.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.214+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(51 gc.6 [call] v31'29 uv29 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.214+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc800 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.214+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(52 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.214+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.214+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(53 gc.6 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:54 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(54 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:55 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(55 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.215+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:56 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd8ac9400 2024-04-27T05:21:56.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.216+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(56 gc.6 [call] v31'30 uv30 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.216+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-04-27T05:21:56.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.216+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0400 2024-04-27T05:21:56.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.217+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(57 gc.7 [call] v31'39 uv39 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.217+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd7fb0400 2024-04-27T05:21:56.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.217+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(58 gc.7 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.217+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.217+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(59 gc.7 [call out=24b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:60 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(60 gc.7 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:61 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(61 gc.7 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.218+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.219+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(62 gc.7 [call] v31'40 uv40 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.219+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-04-27T05:21:56.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.219+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(63 gc.8 [call] v31'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(64 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(65 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.220+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.221+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(66 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.221+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.221+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb1800 2024-04-27T05:21:56.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.221+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(67 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.221+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.221+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd7fb1800 2024-04-27T05:21:56.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.222+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(68 gc.8 [call] v31'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.222+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-04-27T05:21:56.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.222+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.223+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(69 gc.9 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.223+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.223+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(70 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.223+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.223+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(71 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:72 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(72 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(73 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.224+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:74 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.225+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(74 gc.9 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.225+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-04-27T05:21:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.225+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd7fb1800 2024-04-27T05:21:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(75 gc.10 [call] v31'58 uv58 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:76 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd7fb1800 2024-04-27T05:21:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(76 gc.10 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:77 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fddc00 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(77 gc.10 [call out=24b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(78 gc.10 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.226+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:79 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc800 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.227+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 24 ==== osd_op_reply(79 gc.10 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.227+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.227+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb1800 2024-04-27T05:21:56.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.228+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 25 ==== osd_op_reply(80 gc.10 [call] v31'59 uv59 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.228+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-04-27T05:21:56.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.228+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(81 gc.11 [call] v31'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(82 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(83 gc.11 [call out=24b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(84 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.229+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.230+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(85 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.230+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.230+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:86 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd7fb0400 2024-04-27T05:21:56.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.230+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(86 gc.11 [call] v31'25 uv25 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.231+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-04-27T05:21:56.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.231+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:87 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb1800 2024-04-27T05:21:56.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.231+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 26 ==== osd_op_reply(87 gc.12 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.231+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:88 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 27 ==== osd_op_reply(88 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:89 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 28 ==== osd_op_reply(89 gc.12 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:90 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 29 ==== osd_op_reply(90 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:91 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 30 ==== osd_op_reply(91 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.232+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:92 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.233+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 31 ==== osd_op_reply(92 gc.12 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.233+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-04-27T05:21:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.233+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:93 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.234+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(93 gc.13 [call] v31'56 uv56 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.234+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:94 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.234+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(94 gc.13 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.234+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.234+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:95 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(95 gc.13 [call out=24b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:96 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fddc00 con 0x55fdd7fb0400 2024-04-27T05:21:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(96 gc.13 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:97 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd7fb0400 2024-04-27T05:21:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(97 gc.13 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.235+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:98 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc800 con 0x55fdd7fb0400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.236+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(98 gc.13 [call] v31'57 uv57 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.236+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.236+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:99 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.237+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(99 gc.14 [call] v31'169 uv169 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.237+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:100 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.237+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(100 gc.14 [call out=15b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.237+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.237+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:101 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(101 gc.14 [call out=24b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:102 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(102 gc.14 [call out=15b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:103 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd8ac9400 2024-04-27T05:21:56.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(103 gc.14 [call out=15b] v0'0 uv169 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.238+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:104 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd8ac9400 2024-04-27T05:21:56.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.239+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(104 gc.14 [call] v31'170 uv170 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.239+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-04-27T05:21:56.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.239+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:105 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.240+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(105 gc.15 [call] v31'50 uv50 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.240+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:106 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(106 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:107 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(107 gc.15 [call out=24b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:108 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(108 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:109 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 29 ==== osd_op_reply(109 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.241+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:110 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.242+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 30 ==== osd_op_reply(110 gc.15 [call] v31'51 uv51 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.242+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.242+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:111 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046000 con 0x55fdd7fb1800 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.243+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 32 ==== osd_op_reply(111 gc.16 [call] v31'60 uv60 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.243+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:112 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046400 con 0x55fdd7fb1800 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 33 ==== osd_op_reply(112 gc.16 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:113 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046800 con 0x55fdd7fb1800 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 34 ==== osd_op_reply(113 gc.16 [call out=24b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:114 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 35 ==== osd_op_reply(114 gc.16 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:115 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047000 con 0x55fdd7fb1800 2024-04-27T05:21:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 36 ==== osd_op_reply(115 gc.16 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.244+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:116 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047400 con 0x55fdd7fb1800 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.245+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 37 ==== osd_op_reply(116 gc.16 [call] v31'61 uv61 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.245+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.245+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:117 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.246+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 31 ==== osd_op_reply(117 gc.17 [call] v31'32 uv32 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.246+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:118 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.246+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 32 ==== osd_op_reply(118 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.246+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.246+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:119 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 33 ==== osd_op_reply(119 gc.17 [call out=24b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:120 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 34 ==== osd_op_reply(120 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:121 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 35 ==== osd_op_reply(121 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.247+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:122 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.248+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 36 ==== osd_op_reply(122 gc.17 [call] v31'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.248+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.248+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:123 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.249+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(123 gc.18 [call] v31'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.249+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:124 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(124 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:125 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(125 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:126 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(126 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:127 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd8ac9400 2024-04-27T05:21:56.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(127 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.250+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:128 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd8ac9400 2024-04-27T05:21:56.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.251+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(128 gc.18 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.251+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.251+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:129 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.252+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(129 gc.19 [call] v31'171 uv171 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.252+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:130 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.252+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(130 gc.19 [call out=15b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.252+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.252+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:131 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(131 gc.19 [call out=24b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:132 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(132 gc.19 [call out=15b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:133 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(133 gc.19 [call out=15b] v0'0 uv171 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.253+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:134 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd8ac9400 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.254+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 36 ==== osd_op_reply(134 gc.19 [call] v31'172 uv172 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.254+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.254+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:135 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 37 ==== osd_op_reply(135 gc.20 [call] v31'52 uv52 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:136 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 38 ==== osd_op_reply(136 gc.20 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:137 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 39 ==== osd_op_reply(137 gc.20 [call out=24b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.255+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:138 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.256+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 40 ==== osd_op_reply(138 gc.20 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.256+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.256+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:139 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.256+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 41 ==== osd_op_reply(139 gc.20 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.256+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.256+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:140 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.257+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 42 ==== osd_op_reply(140 gc.20 [call] v31'53 uv53 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.257+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-04-27T05:21:56.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.257+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:141 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046400 con 0x55fdd7fb1800 2024-04-27T05:21:56.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.258+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 38 ==== osd_op_reply(141 gc.21 [call] v31'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.258+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:142 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046800 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.258+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 39 ==== osd_op_reply(142 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.258+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.258+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:143 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 40 ==== osd_op_reply(143 gc.21 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:144 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047000 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 41 ==== osd_op_reply(144 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:145 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047400 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 42 ==== osd_op_reply(145 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.259+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:146 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd7fb1800 2024-04-27T05:21:56.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.260+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 43 ==== osd_op_reply(146 gc.21 [call] v31'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.260+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-04-27T05:21:56.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.260+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:147 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 43 ==== osd_op_reply(147 gc.22 [call] v31'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:148 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 44 ==== osd_op_reply(148 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:149 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 45 ==== osd_op_reply(149 gc.22 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.261+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:150 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.262+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 46 ==== osd_op_reply(150 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.262+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.262+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:151 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.262+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 47 ==== osd_op_reply(151 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.262+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.262+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:152 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.263+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 48 ==== osd_op_reply(152 gc.22 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.263+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.263+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:153 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 49 ==== osd_op_reply(153 gc.23 [call] v31'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:154 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 50 ==== osd_op_reply(154 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efe900 con 0x55fdd7fb0c00 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:155 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 51 ==== osd_op_reply(155 gc.23 [call out=24b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:156 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 52 ==== osd_op_reply(156 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.264+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:157 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.265+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 53 ==== osd_op_reply(157 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.265+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.265+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:158 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.265+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 54 ==== osd_op_reply(158 gc.23 [call] v31'39 uv39 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.265+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.266+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:159 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.266+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 55 ==== osd_op_reply(159 gc.24 [call] v31'22 uv22 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.266+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:160 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 56 ==== osd_op_reply(160 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:161 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 57 ==== osd_op_reply(161 gc.24 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:162 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 58 ==== osd_op_reply(162 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:163 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 59 ==== osd_op_reply(163 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.267+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:164 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.268+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 60 ==== osd_op_reply(164 gc.24 [call] v31'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.268+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-04-27T05:21:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.268+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:165 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 61 ==== osd_op_reply(165 gc.25 [call] v31'50 uv50 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:166 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 62 ==== osd_op_reply(166 gc.25 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:167 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 63 ==== osd_op_reply(167 gc.25 [call out=24b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.269+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:168 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.270+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 64 ==== osd_op_reply(168 gc.25 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.270+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.270+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.270+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 65 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.270+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.270+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.271+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 66 ==== osd_op_reply(170 gc.25 [call] v31'51 uv51 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.271+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-04-27T05:21:56.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.271+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:171 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.272+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(171 gc.26 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.272+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb0400 2024-04-27T05:21:56.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.272+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd7fb0400 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb0400 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:175 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== osd_op_reply(175 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.273+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:176 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb0400 2024-04-27T05:21:56.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.275+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== osd_op_reply(176 gc.26 [call] v31'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.275+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-04-27T05:21:56.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.275+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 67 ==== osd_op_reply(177 gc.27 [call] v31'32 uv32 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:178 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 68 ==== osd_op_reply(178 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:179 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 69 ==== osd_op_reply(179 gc.27 [call out=24b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.276+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:180 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fddc00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.277+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 70 ==== osd_op_reply(180 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.277+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.277+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:181 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.277+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 71 ==== osd_op_reply(181 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.277+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.277+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:182 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.278+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 72 ==== osd_op_reply(182 gc.27 [call] v31'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.278+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-04-27T05:21:56.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.278+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:183 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== osd_op_reply(183 gc.28 [call] v31'41 uv41 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:184 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 33 ==== osd_op_reply(184 gc.28 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:185 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 34 ==== osd_op_reply(185 gc.28 [call out=24b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.279+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:186 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.280+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 35 ==== osd_op_reply(186 gc.28 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.280+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.280+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:187 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.280+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 36 ==== osd_op_reply(187 gc.28 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.280+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.280+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:188 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb0400 2024-04-27T05:21:56.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.281+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 37 ==== osd_op_reply(188 gc.28 [call] v31'42 uv42 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.281+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-04-27T05:21:56.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.281+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:189 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 73 ==== osd_op_reply(189 gc.29 [call] v31'54 uv54 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:190 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 74 ==== osd_op_reply(190 gc.29 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:191 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdd800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 75 ==== osd_op_reply(191 gc.29 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:192 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdc400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.282+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 76 ==== osd_op_reply(192 gc.29 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.283+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.283+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:193 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fdcc00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.283+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 77 ==== osd_op_reply(193 gc.29 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.283+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.283+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:194 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.284+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 78 ==== osd_op_reply(194 gc.29 [call] v31'55 uv55 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.284+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-04-27T05:21:56.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.284+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:195 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 38 ==== osd_op_reply(195 gc.30 [call] v31'58 uv58 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:196 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046000 con 0x55fdd7fb0400 2024-04-27T05:21:56.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 39 ==== osd_op_reply(196 gc.30 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:197 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0400 2024-04-27T05:21:56.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 40 ==== osd_op_reply(197 gc.30 [call out=24b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.285+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:198 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd7fb0400 2024-04-27T05:21:56.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.286+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 41 ==== osd_op_reply(198 gc.30 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.286+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.286+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:199 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.286+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 42 ==== osd_op_reply(199 gc.30 [call out=15b] v0'0 uv58 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.286+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.286+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:200 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb0400 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.287+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 43 ==== osd_op_reply(200 gc.30 [call] v31'59 uv59 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.287+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-04-27T05:21:56.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.287+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:201 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 44 ==== osd_op_reply(201 gc.31 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:202 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 45 ==== osd_op_reply(202 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:203 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 46 ==== osd_op_reply(203 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:204 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 47 ==== osd_op_reply(204 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:205 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027000 con 0x55fdd7fb1800 2024-04-27T05:21:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 48 ==== osd_op_reply(205 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.288+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:206 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1c00 con 0x55fdd7fb1800 2024-04-27T05:21:56.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.289+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.2 v2:172.21.15.138:6809/29865 49 ==== osd_op_reply(206 gc.31 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7fb1800 2024-04-27T05:21:56.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.289+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-04-27T05:21:56.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.289+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:207 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1000 con 0x55fdd8ac9400 2024-04-27T05:21:56.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.290+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 37 ==== osd_op_reply(207 gc.0 [call] v31'54 uv54 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.290+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:208 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0800 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 38 ==== osd_op_reply(208 gc.0 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:209 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026000 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 39 ==== osd_op_reply(209 gc.0 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:210 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027c00 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 40 ==== osd_op_reply(210 gc.0 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:211 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026400 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 41 ==== osd_op_reply(211 gc.0 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.291+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:212 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026800 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.292+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 42 ==== osd_op_reply(212 gc.0 [call] v31'55 uv55 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.292+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-04-27T05:21:56.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.292+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:213 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.293+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 44 ==== osd_op_reply(213 gc.1 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.293+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:214 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046400 con 0x55fdd7fb0400 2024-04-27T05:21:56.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.293+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 45 ==== osd_op_reply(214 gc.1 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.293+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.293+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:215 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046800 con 0x55fdd7fb0400 2024-04-27T05:21:56.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 46 ==== osd_op_reply(215 gc.1 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:216 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 47 ==== osd_op_reply(216 gc.1 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:217 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047000 con 0x55fdd7fb0400 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 48 ==== osd_op_reply(217 gc.1 [call out=1702b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+1702 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 20 garbage collection: RGWGC::process iterating over entry tag='2f19b8ff-2608-4d86-8f1c-430623205476.4229.15701436620086009928', time=2024-04-27T05:21:55.232074+0000, chain.objs.size()=7 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_1 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:218 3.1b 3:db7a5c70:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_2 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:219 3.11 3:8b75df32:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_3 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:220 3.2 3:406c9c3d:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_4 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:221 3.17 3:e8f07ef4:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0ec00 con 0x55fdd8ac9400 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_5 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:222 3.27 3:e518f4d9:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd7fb0400 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_6 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:223 3.33 3:cf772884:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8963400 con 0x55fdd7fb0400 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_7 2024-04-27T05:21:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.294+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:224 3.32 3:4cc86049:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.297+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 49 ==== osd_op_reply(223 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_6 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.297+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 79 ==== osd_op_reply(224 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_7 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.297+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 80 ==== osd_op_reply(218 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_1 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.298+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 50 ==== osd_op_reply(222 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_5 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.299+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 <== osd.1 v2:172.21.15.138:6800/29864 43 ==== osd_op_reply(221 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_4 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd8ac9400 2024-04-27T05:21:56.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.300+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 81 ==== osd_op_reply(220 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_3 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.300+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 82 ==== osd_op_reply(219 2f19b8ff-2608-4d86-8f1c-430623205476.4231.1__shadow_.CrovUJMjE9gBwjzdhaK9finBJTnY_NJ_2 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.300+0000 7f80cbd2a880 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-27T05:21:56.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.300+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:225 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047c00 con 0x55fdd7fb0400 2024-04-27T05:21:56.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.301+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 51 ==== osd_op_reply(225 gc.1 [call] v31'28 uv28 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.301+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:226 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046400 con 0x55fdd7fb0400 2024-04-27T05:21:56.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.304+0000 7f80c7e4c640 1 -- 172.21.15.138:0/3986990452 <== osd.0 v2:172.21.15.114:6800/27747 52 ==== osd_op_reply(226 gc.1 [call] v31'29 uv29 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7fb0400 2024-04-27T05:21:56.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.304+0000 7f80cbd2a880 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-04-27T05:21:56.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.304+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:227 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.305+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 83 ==== osd_op_reply(227 gc.2 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.305+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:228 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.305+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 84 ==== osd_op_reply(228 gc.2 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.305+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.305+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:229 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047000 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 85 ==== osd_op_reply(229 gc.2 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:230 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027800 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 86 ==== osd_op_reply(230 gc.2 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:231 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8026c00 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 87 ==== osd_op_reply(231 gc.2 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80cbd2a880 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T05:21:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.306+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:232 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb1400 con 0x55fdd7fb0c00 2024-04-27T05:21:56.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.307+0000 7f80c8e4e640 1 -- 172.21.15.138:0/3986990452 <== osd.3 v2:172.21.15.138:6808/29866 88 ==== osd_op_reply(232 gc.2 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7fb0c00 2024-04-27T05:21:56.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.307+0000 7f80af61b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.307+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94548737158400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7fb0000 con 0x55fdd8963000 2024-04-27T05:21:56.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.308+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94548737158400] v31'173 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd8963000 2024-04-27T05:21:56.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.308+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94548737149184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047800 con 0x55fdd7eef000 2024-04-27T05:21:56.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.309+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94548737149184] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.309+0000 7f80cbd2a880 20 remove_watcher() i=0 2024-04-27T05:21:56.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.309+0000 7f80cbd2a880 2 removed watcher, disabling cache 2024-04-27T05:21:56.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.309+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94548737150336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8047400 con 0x55fdd8963000 2024-04-27T05:21:56.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.310+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94548737150336] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd8963000 2024-04-27T05:21:56.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.310+0000 7f80cbd2a880 20 remove_watcher() i=1 2024-04-27T05:21:56.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.310+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94548737151488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8046000 con 0x55fdd7eef800 2024-04-27T05:21:56.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.311+0000 7f80c864d640 1 -- 172.21.15.138:0/2483192203 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94548737151488] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efefc0 con 0x55fdd7eef800 2024-04-27T05:21:56.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.312+0000 7f80cbd2a880 20 remove_watcher() i=2 2024-04-27T05:21:56.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.312+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94548737152640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0e400 con 0x55fdd7eef000 2024-04-27T05:21:56.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.312+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94548737152640] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7eff200 con 0x55fdd7eef000 2024-04-27T05:21:56.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.312+0000 7f80cbd2a880 20 remove_watcher() i=3 2024-04-27T05:21:56.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.312+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94548737153792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd7f0f000 con 0x55fdd8963000 2024-04-27T05:21:56.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.313+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94548737153792] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd8963000 2024-04-27T05:21:56.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.313+0000 7f80cbd2a880 20 remove_watcher() i=4 2024-04-27T05:21:56.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.313+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94548737154944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8027400 con 0x55fdd7eefc00 2024-04-27T05:21:56.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.314+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94548737154944] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7eefc00 2024-04-27T05:21:56.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.314+0000 7f80cbd2a880 20 remove_watcher() i=5 2024-04-27T05:21:56.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.314+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94548737156096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8aea000 con 0x55fdd7eefc00 2024-04-27T05:21:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.315+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94548737156096] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7eefc00 2024-04-27T05:21:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.315+0000 7f80cbd2a880 20 remove_watcher() i=6 2024-04-27T05:21:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.315+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94548737157248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55fdd8aea400 con 0x55fdd7eefc00 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80c8e4e640 1 -- 172.21.15.138:0/2483192203 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94548737157248] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55fdd7efeb40 con 0x55fdd7eefc00 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 20 remove_watcher() i=7 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7eefc00 msgr2=0x55fdd7e4f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7eefc00 0x55fdd7e4f180 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8963000 msgr2=0x55fdd7e4db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8963000 0x55fdd7e4db80 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7eef000 msgr2=0x55fdd7e4ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7eef000 0x55fdd7e4ec00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7eef800 msgr2=0x55fdd7e4e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7eef800 0x55fdd7e4e100 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd7eee000 msgr2=0x55fdd7e4d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd7eee000 0x55fdd7e4d600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55fdd8ad9260 tx=0x55fdd8adaf60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 msgr2=0x55fdd7e4cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a26e70 tx=0x55fdd7c7da40 comp rx=0 tx=0).stop 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80c7e4c640 1 -- 172.21.15.138:0/2483192203 reap_dead start 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 shutdown_connections 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/2483192203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd8962c00 0x55fdd7e4cb00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.316+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 >> 172.21.15.138:0/2483192203 conn(0x55fdd7b6b400 msgr2=0x55fdd7c75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 shutdown_connections 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/2483192203 wait complete. 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7fb0400 msgr2=0x55fdd7fac580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55fdd7fb0400 0x55fdd7fac580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8ac9400 msgr2=0x55fdd7fac000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55fdd8ac9400 0x55fdd7fac000 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7fb1800 msgr2=0x55fdd7fad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55fdd7fb1800 0x55fdd7fad080 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7fb0c00 msgr2=0x55fdd7facb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55fdd7fb0c00 0x55fdd7facb00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8ac9800 msgr2=0x55fdd7e4f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8ac9800 0x55fdd7e4f700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55fdd7ecb200 tx=0x55fdd8adbe60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 msgr2=0x55fdd7e4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80c864d640 1 -- 172.21.15.138:0/3986990452 reap_dead start 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55fdd7ecb830 tx=0x55fdd8adb080 comp rx=0 tx=0).stop 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 shutdown_connections 2024-04-27T05:21:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3986990452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7eeec00 0x55fdd7e4e680 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 >> 172.21.15.138:0/3986990452 conn(0x55fdd7f0f800 msgr2=0x55fdd7c75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 shutdown_connections 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3986990452 wait complete. 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8963c00 msgr2=0x55fdd7e4d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8963c00 0x55fdd7e4d080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a6a2d0 tx=0x55fdd7c7dc20 comp rx=0 tx=0).stop 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 msgr2=0x55fdd7e4c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55fdd8a26de0 tx=0x55fdd7c7cc60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 shutdown_connections 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55fdd8963c00 0x55fdd7e4d080 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 --2- 172.21.15.138:0/3508368191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fdd7b6b000 0x55fdd7e4c580 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.317+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 >> 172.21.15.138:0/3508368191 conn(0x55fdd7b6ac00 msgr2=0x55fdd7c73000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.318+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 shutdown_connections 2024-04-27T05:21:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.318+0000 7f80cbd2a880 1 -- 172.21.15.138:0/3508368191 wait complete. 2024-04-27T05:21:56.324 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-27T05:21:56.324 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-27T05:21:56.325 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-27T05:21:56.408 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:56.408 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:56.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.407+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.407+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.407+0000 7f3000435880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ca17637400 0x55ca17918580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.407+0000 7f3000435880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ca1790e600 con 0x55ca17637000 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.407+0000 7f3000435880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ca17676280 con 0x55ca17637400 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.408+0000 7f2ffbbf3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55ca17637400 0x55ca17918580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.408+0000 7f2ffc3f4640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55ca17637000 0x55ca1773e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45024/0 (socket says 172.21.15.138:45024) 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.408+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1348077961 learned_addr learned my addr 172.21.15.138:0/1348077961 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.408+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1348077961 >> v1:172.21.15.114:6789/0 conn(0x55ca17637000 legacy=0x55ca1773e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.408+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1348077961 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1763b380 con 0x55ca17637400 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/1348077961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637400 0x55ca17918580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55ca17742c00 tx=0x55ca17748de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a63dfa85c528efd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f2ffb3f2640 1 -- 172.21.15.138:0/1348077961 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764fce0 con 0x55ca17637400 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f2ffb3f2640 1 -- 172.21.15.138:0/1348077961 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca1790fb00 con 0x55ca17637400 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f2ffb3f2640 1 -- 172.21.15.138:0/1348077961 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764fe40 con 0x55ca17637400 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 -- 172.21.15.138:0/1348077961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637400 msgr2=0x55ca17918580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 --2- 172.21.15.138:0/1348077961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637400 0x55ca17918580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55ca17742c00 tx=0x55ca17748de0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 -- 172.21.15.138:0/1348077961 shutdown_connections 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 --2- 172.21.15.138:0/1348077961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637400 0x55ca17918580 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 -- 172.21.15.138:0/1348077961 >> 172.21.15.138:0/1348077961 conn(0x55ca17636c00 msgr2=0x55ca1773e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 -- 172.21.15.138:0/1348077961 shutdown_connections 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.410+0000 7f3000435880 1 -- 172.21.15.138:0/1348077961 wait complete. 2024-04-27T05:21:56.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f3000435880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f3000435880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17676780 con 0x55ca17637000 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f2ffc3f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f2ffc3f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44740/0 (socket says 172.21.15.138:44740) 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.411+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1142024299 learned_addr learned my addr 172.21.15.138:0/1142024299 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1142024299 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1763aea0 con 0x55ca17637000 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/1142024299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55ca184f2cc0 tx=0x55ca17748cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb5ad67f84211de server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f2ffabf1640 1 -- 172.21.15.138:0/1142024299 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764f4a0 con 0x55ca17637000 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f2ffabf1640 1 -- 172.21.15.138:0/1142024299 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca1790fe00 con 0x55ca17637000 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f2ffabf1640 1 -- 172.21.15.138:0/1142024299 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764fce0 con 0x55ca17637000 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 -- 172.21.15.138:0/1142024299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 msgr2=0x55ca17918580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 --2- 172.21.15.138:0/1142024299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55ca184f2cc0 tx=0x55ca17748cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 -- 172.21.15.138:0/1142024299 shutdown_connections 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 --2- 172.21.15.138:0/1142024299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 -- 172.21.15.138:0/1142024299 >> 172.21.15.138:0/1142024299 conn(0x55ca17636c00 msgr2=0x55ca1773f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 -- 172.21.15.138:0/1142024299 shutdown_connections 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.412+0000 7f3000435880 1 -- 172.21.15.138:0/1142024299 wait complete. 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f3000435880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f3000435880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca176768c0 con 0x55ca17637000 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f2ffc3f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f2ffc3f4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44752/0 (socket says 172.21.15.138:44752) 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/2434464769 learned_addr learned my addr 172.21.15.138:0/2434464769 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/2434464769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca1763b6c0 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55ca184f2ed0 tx=0x55ca17748c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.413+0000 7f2ff93ee640 1 -- 172.21.15.138:0/2434464769 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764fe40 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ff93ee640 1 -- 172.21.15.138:0/2434464769 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca18508000 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca1763b1e0 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ff93ee640 1 -- 172.21.15.138:0/2434464769 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764f4a0 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca176a8ea0 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f3000435880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f3000435880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca17676c80 con 0x55ca1842f800 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ff93ee640 1 -- 172.21.15.138:0/2434464769 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca17661600 con 0x55ca17637000 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ffcbf5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ffcbf5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44760/0 (socket says 172.21.15.138:44760) 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/3141822105 learned_addr learned my addr 172.21.15.138:0/3141822105 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ff93ee640 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca18598000 0x55ca17919080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.414+0000 7f2ff93ee640 1 -- 172.21.15.138:0/2434464769 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55ca17649a40 con 0x55ca17637000 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca18598000 0x55ca17919080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/3141822105 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca176a8d00 con 0x55ca1842f800 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/3141822105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55ca18543170 tx=0x55ca17749c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e805809362666545 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca18598000 0x55ca17919080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ca185421b0 tx=0x55ca177494a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ff6be9640 1 -- 172.21.15.138:0/3141822105 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764fe40 con 0x55ca1842f800 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ff6be9640 1 -- 172.21.15.138:0/3141822105 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca18508300 con 0x55ca1842f800 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f2ff6be9640 1 -- 172.21.15.138:0/3141822105 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764f4a0 con 0x55ca1842f800 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 -- 172.21.15.138:0/3141822105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 msgr2=0x55ca17918b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 --2- 172.21.15.138:0/3141822105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55ca18543170 tx=0x55ca17749c80 comp rx=0 tx=0).stop 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 -- 172.21.15.138:0/3141822105 shutdown_connections 2024-04-27T05:21:56.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 --2- 172.21.15.138:0/3141822105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 -- 172.21.15.138:0/3141822105 >> 172.21.15.138:0/3141822105 conn(0x55ca17637400 msgr2=0x55ca17741000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 -- 172.21.15.138:0/3141822105 shutdown_connections 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.415+0000 7f3000435880 1 -- 172.21.15.138:0/3141822105 wait complete. 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f3000435880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f3000435880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1765a280 con 0x55ca1842f800 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ffcbf5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ffcbf5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44770/0 (socket says 172.21.15.138:44770) 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 learned_addr learned my addr 172.21.15.138:0/1850371200 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca176a8b60 con 0x55ca1842f800 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55ca184f39e0 tx=0x55ca17749aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ff53e6640 1 -- 172.21.15.138:0/1850371200 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764fe40 con 0x55ca1842f800 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ff53e6640 1 -- 172.21.15.138:0/1850371200 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca18508480 con 0x55ca1842f800 2024-04-27T05:21:56.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca176a89c0 con 0x55ca1842f800 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.416+0000 7f2ff53e6640 1 -- 172.21.15.138:0/1850371200 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1764f4a0 con 0x55ca1842f800 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.417+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca176a8820 con 0x55ca1842f800 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.417+0000 7f2ff53e6640 1 -- 172.21.15.138:0/1850371200 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca176618c0 con 0x55ca1842f800 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ff53e6640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca179c4000 0x55ca17919600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ff53e6640 1 -- 172.21.15.138:0/1850371200 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55ca17649c00 con 0x55ca1842f800 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca179c4000 0x55ca17919600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842fc00 0x55ca17919b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4400 con 0x55ca1842fc00 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842fc00 0x55ca17919b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca179c4000 0x55ca17919600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55ca17742c00 tx=0x55ca185b8d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842fc00 0x55ca17919b80 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 20 realm 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.418+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4800 con 0x55ca1842fc00 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4c00 con 0x55ca1842fc00 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5000 con 0x55ca1842fc00 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca179c5400 0x55ca1791a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4400 con 0x55ca179c5400 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca179c5400 0x55ca1791a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.419+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca179c5400 0x55ca1791a100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f3000435880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f3000435880 20 searching for the correct realm 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4800 con 0x55ca179c5400 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca179c4800 0x55ca1791ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4400 con 0x55ca179c4800 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca179c4800 0x55ca1791ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.420+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca179c4800 0x55ca1791ac00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.421+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.421+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca179c4400 0x55ca1791b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.421+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.421+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca179c4400 0x55ca1791b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.421+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca179c4400 0x55ca1791b180 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.422+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4800 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.423+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4800 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.424+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.425+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4800 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.426+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4800 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c5400 2024-04-27T05:21:56.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4400 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.427+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca1842fc00 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1842e800 con 0x55ca179c4800 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4c00 con 0x55ca1842fc00 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5000 con 0x55ca1842fc00 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5800 con 0x55ca179c5400 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.428+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 zone default found 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 4 Realm: () 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 zonegroup default 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca179c4400 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca179c5400 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca179c5400 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 started sync module instance, tier type = 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:56.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.429+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a800 con 0x55ca179c4800 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.430+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.430+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94326467135744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841ac00 con 0x55ca179c4800 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.430+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841b000 con 0x55ca1842fc00 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.431+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.431+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94326467135744] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.431+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94326467136896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841b400 con 0x55ca1842fc00 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.431+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841b800 con 0x55ca179c5400 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.432+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94326467136896] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.433+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.433+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94326467138048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca179c5400 2024-04-27T05:21:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.433+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca179c4800 2024-04-27T05:21:56.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.434+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94326467138048] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.434+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.434+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94326467139200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841b800 con 0x55ca179c4800 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.434+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5800 con 0x55ca1842fc00 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.435+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.435+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94326467139200] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.435+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94326467140352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c4c00 con 0x55ca1842fc00 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.435+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5000 con 0x55ca179c4400 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.436+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.436+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94326467141504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca179c4400 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.436+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca18599800 con 0x55ca179c4400 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.436+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94326467140352] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.437+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94326467141504] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.437+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.437+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94326467142656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a800 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.437+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841ac00 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94326467142656] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94326467143808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=0 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=1 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=2 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=3 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=4 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=5 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.438+0000 7f3000435880 20 add_watcher() i=6 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.439+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94326467143808] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.439+0000 7f3000435880 20 add_watcher() i=7 2024-04-27T05:21:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.439+0000 7f3000435880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.439+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841b800 con 0x55ca1842fc00 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.440+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.440+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94326467144960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841b000 con 0x55ca1842fc00 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94326467144960] v31'174 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f2fe33c2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f3000435880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f3000435880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f3000435880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.441+0000 7f3000435880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1765a8c0 con 0x55ca1841bc00 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2ffbbf3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2ffbbf3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44776/0 (socket says 172.21.15.138:44776) 2024-04-27T05:21:56.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/2674045837 learned_addr learned my addr 172.21.15.138:0/2674045837 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/2674045837 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca176a8680 con 0x55ca1841bc00 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/2674045837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55ca18540e70 tx=0x55ca185b8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbe3178184cbd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2fe2bc1640 1 -- 172.21.15.138:0/2674045837 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1841c580 con 0x55ca1841bc00 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2fe2bc1640 1 -- 172.21.15.138:0/2674045837 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca18508000 con 0x55ca1841bc00 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.442+0000 7f2fe2bc1640 1 -- 172.21.15.138:0/2674045837 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1841c6e0 con 0x55ca1841bc00 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- 172.21.15.138:0/2674045837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 msgr2=0x55ca1791a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 --2- 172.21.15.138:0/2674045837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55ca18540e70 tx=0x55ca185b8de0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- 172.21.15.138:0/2674045837 shutdown_connections 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 --2- 172.21.15.138:0/2674045837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- 172.21.15.138:0/2674045837 >> 172.21.15.138:0/2674045837 conn(0x55ca1841b400 msgr2=0x55ca17741800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- 172.21.15.138:0/2674045837 shutdown_connections 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- 172.21.15.138:0/2674045837 wait complete. 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 Processor -- start 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- start start 2024-04-27T05:21:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.443+0000 7f3000435880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca1765a780 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2ffbbf3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2ffbbf3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44784/0 (socket says 172.21.15.138:44784) 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 learned_addr learned my addr 172.21.15.138:0/4121050682 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca176a84e0 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55ca179a3710 tx=0x55ca185b8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2fe13be640 1 -- 172.21.15.138:0/4121050682 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1841c840 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2fe13be640 1 -- 172.21.15.138:0/4121050682 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca18508780 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca1763aea0 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f2fe13be640 1 -- 172.21.15.138:0/4121050682 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca1841c580 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.444+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca1763b6c0 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2fe13be640 1 -- 172.21.15.138:0/4121050682 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55ca17660b00 con 0x55ca1841bc00 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2fe13be640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca1841a800 0x55ca1791b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2fe13be640 1 -- 172.21.15.138:0/4121050682 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55ca17649dc0 con 0x55ca1841bc00 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca1841a800 0x55ca1791b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842e800 0x55ca17a7a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca1842e800 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842e800 0x55ca17a7a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca1841a800 0x55ca1791b700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55ca179a2ff0 tx=0x55ca185b90e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.445+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842e800 0x55ca17a7a000 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.446+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'56 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.446+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca17a7e400 0x55ca17a7a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.446+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e800 con 0x55ca17a7e400 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.447+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca17a7e400 0x55ca17a7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.447+0000 7f2ffbbf3640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca17a7e400 0x55ca17a7a580 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.448+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.448+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca17a7ec00 0x55ca17a7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.448+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca17a7ec00 2024-04-27T05:21:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.448+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca17a7ec00 0x55ca17a7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.448+0000 7f2ffcbf5640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca17a7ec00 0x55ca17a7ab00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.449+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.449+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca1842e800 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.450+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'175 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.450+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca17a7f800 0x55ca17a7b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.450+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7f800 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.450+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca17a7f800 0x55ca17a7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.450+0000 7f2ffc3f4640 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca17a7f800 0x55ca17a7b080 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.451+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'62 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.451+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7ec00 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.452+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'52 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.452+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6400 con 0x55ca1842e800 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.454+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'31 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.454+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6800 con 0x55ca17a7e400 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.455+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'43 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.455+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6c00 con 0x55ca17a7f800 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.455+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.455+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7ec00 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.456+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.456+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7400 con 0x55ca17a7f800 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.457+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'63 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.457+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7800 con 0x55ca17a7e400 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.458+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'26 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.458+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7c00 con 0x55ca17a7f800 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.459+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.459+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6400 con 0x55ca17a7e400 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.460+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'60 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.460+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6c00 con 0x55ca1842e800 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.461+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'176 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.461+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca17a7ec00 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.462+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'56 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.462+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca17a7f800 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.463+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'64 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.463+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7ec00 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.464+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'34 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.464+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca1842e800 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.465+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.465+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca1842e800 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.466+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'177 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.466+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.467+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'57 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.467+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7f800 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.468+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'38 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.468+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.469+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.469+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.470+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'40 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.470+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841ac00 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.470+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.470+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca17a7ec00 2024-04-27T05:21:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.471+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'53 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.471+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca17a7e400 2024-04-27T05:21:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.473+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'31 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.473+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca17a7ec00 2024-04-27T05:21:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.473+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'34 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.473+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7e400 2024-04-27T05:21:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.474+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'44 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.474+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca17a7ec00 2024-04-27T05:21:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.475+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'58 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.475+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7400 con 0x55ca17a7e400 2024-04-27T05:21:56.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.476+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'61 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.476+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7c00 con 0x55ca17a7f800 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.477+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.477+0000 7f2fdebb9640 20 reqs_thread_entry: start 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.477+0000 7f3000435880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.477+0000 7f3000435880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.477+0000 7f2fdcbb5640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:26 2024 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f2fdc393640 20 reqs_thread_entry: start 2024-04-27T05:21:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e800 con 0x55ca1842e800 2024-04-27T05:21:56.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca1842e800 2024-04-27T05:21:56.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6800 con 0x55ca1842e800 2024-04-27T05:21:56.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.478+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7800 con 0x55ca17a7e400 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.479+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.479+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6400 con 0x55ca17a7e400 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.479+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.479+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca17a7e400 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.479+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.479+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5000 con 0x55ca17a7ec00 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5000 con 0x55ca17a7ec00 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841ac00 con 0x55ca17a7ec00 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca1842e800 2024-04-27T05:21:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv168 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.480+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca1842e800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv168 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca1842e800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv168 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca17a7f800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7f800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv57 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.481+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7f800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e800 con 0x55ca17a7ec00 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca17a7ec00 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv49 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca17a7ec00 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca1842e800 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.482+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca1842e800 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.483+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.483+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca1842e800 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.483+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.483+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7c00 con 0x55ca17a7e400 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.483+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.483+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6800 con 0x55ca17a7e400 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.484+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv40 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.484+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02000 con 0x55ca17a7e400 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.484+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.484+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02400 con 0x55ca17a7f800 2024-04-27T05:21:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.484+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.484+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02800 con 0x55ca17a7f800 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02c00 con 0x55ca17a7f800 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03000 con 0x55ca17a7ec00 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03400 con 0x55ca17a7ec00 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.485+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03800 con 0x55ca17a7ec00 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03c00 con 0x55ca17a7f800 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02400 con 0x55ca17a7f800 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv59 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02800 con 0x55ca17a7f800 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.486+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02c00 con 0x55ca17a7e400 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.487+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.487+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03c00 con 0x55ca17a7e400 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.487+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.487+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02000 con 0x55ca17a7e400 2024-04-27T05:21:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.487+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.487+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7f800 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7f800 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7400 con 0x55ca17a7f800 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7800 con 0x55ca17a7e400 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.488+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6400 con 0x55ca17a7e400 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7c00 con 0x55ca17a7e400 2024-04-27T05:21:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv57 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6800 con 0x55ca1842e800 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv170 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca1842e800 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv170 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.489+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca1842e800 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.490+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv170 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.490+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7ec00 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.490+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.490+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca17a7ec00 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.490+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.490+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca17a7ec00 2024-04-27T05:21:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e800 con 0x55ca17a7f800 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca17a7f800 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca17a7f800 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv61 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.491+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7ec00 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03000 con 0x55ca17a7ec00 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03400 con 0x55ca17a7ec00 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03800 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841ac00 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.492+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5000 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv172 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e800 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv172 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.493+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f400 con 0x55ca1842e800 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv172 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca1842e800 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca17a7ec00 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7ec00 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 29 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7ec00 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 30 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.494+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7400 con 0x55ca17a7f800 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6800 con 0x55ca17a7f800 2024-04-27T05:21:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 24 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02400 con 0x55ca17a7f800 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 25 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02800 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 31 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.495+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b08000 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 32 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b08400 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 33 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b08800 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 34 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b08c00 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 35 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b09000 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 36 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.496+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b09400 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 37 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b09800 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 38 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b09c00 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 39 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b08000 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 40 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03000 con 0x55ca17a7ec00 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 41 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.497+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03400 con 0x55ca17a7ec00 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.498+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 42 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.498+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02800 con 0x55ca17a7e400 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.498+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.498+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7e400 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.498+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.498+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7e400 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.499+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.499+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca17a7ec00 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.499+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 43 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.499+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7fc00 con 0x55ca17a7ec00 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 44 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a400 con 0x55ca17a7ec00 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 45 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca1841a000 con 0x55ca17a7e400 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7800 con 0x55ca17a7e400 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.500+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6400 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7c00 con 0x55ca17a7ec00 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 46 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6000 con 0x55ca17a7ec00 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 47 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7000 con 0x55ca17a7ec00 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/4121050682 <== osd.3 v2:172.21.15.138:6808/29866 48 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca17a7ec00 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.501+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03000 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.502+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.502+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03400 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.502+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv59 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.502+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02c00 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.502+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/4121050682 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv59 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7440 con 0x55ca17a7e400 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.503+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03c00 con 0x55ca17a7f800 2024-04-27T05:21:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.503+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 26 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.503+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02000 con 0x55ca17a7f800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.503+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 27 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.503+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02800 con 0x55ca17a7f800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.503+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 <== osd.2 v2:172.21.15.138:6809/29865 28 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55ca179d7200 con 0x55ca17a7f800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.504+0000 7f2fe33c2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.504+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94326467144960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca179c5c00 con 0x55ca1842fc00 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.505+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94326467144960] v31'178 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.505+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94326467135744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03800 con 0x55ca179c4800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.506+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94326467135744] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.506+0000 7f3000435880 20 remove_watcher() i=0 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.506+0000 7f3000435880 2 removed watcher, disabling cache 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.506+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94326467136896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b02400 con 0x55ca1842fc00 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.507+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94326467136896] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.507+0000 7f3000435880 20 remove_watcher() i=1 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.507+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94326467138048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17b03c00 con 0x55ca179c5400 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.508+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/1850371200 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94326467138048] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7200 con 0x55ca179c5400 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.508+0000 7f3000435880 20 remove_watcher() i=2 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.508+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94326467139200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa7400 con 0x55ca179c4800 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.509+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94326467139200] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d7440 con 0x55ca179c4800 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.509+0000 7f3000435880 20 remove_watcher() i=3 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.509+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94326467140352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17aa6800 con 0x55ca1842fc00 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.510+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94326467140352] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca1842fc00 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.510+0000 7f3000435880 20 remove_watcher() i=4 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.510+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94326467141504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e000 con 0x55ca179c4400 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.511+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94326467141504] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.511+0000 7f3000435880 20 remove_watcher() i=5 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.511+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94326467142656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7f000 con 0x55ca179c4400 2024-04-27T05:21:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.511+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94326467142656] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.511+0000 7f3000435880 20 remove_watcher() i=6 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.511+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94326467143808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ca17a7e800 con 0x55ca179c4400 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f2ffcbf5640 1 -- 172.21.15.138:0/1850371200 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94326467143808] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca179d6b40 con 0x55ca179c4400 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 20 remove_watcher() i=7 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca179c4400 msgr2=0x55ca1791b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca179c4400 0x55ca1791b180 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842fc00 msgr2=0x55ca17919b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842fc00 0x55ca17919b80 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca179c4800 msgr2=0x55ca1791ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.512+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca179c4800 0x55ca1791ac00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca179c5400 msgr2=0x55ca1791a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca179c5400 0x55ca1791a100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca179c4000 msgr2=0x55ca17919600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca179c4000 0x55ca17919600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55ca17742c00 tx=0x55ca185b8d80 comp rx=0 tx=0).stop 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 msgr2=0x55ca17918b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55ca184f39e0 tx=0x55ca17749aa0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f2ffbbf3640 1 -- 172.21.15.138:0/1850371200 reap_dead start 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 shutdown_connections 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/1850371200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1842f800 0x55ca17918b00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 >> 172.21.15.138:0/1850371200 conn(0x55ca17637400 msgr2=0x55ca17741000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 shutdown_connections 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/1850371200 wait complete. 2024-04-27T05:21:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca17a7e400 msgr2=0x55ca17a7a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55ca17a7e400 0x55ca17a7a580 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842e800 msgr2=0x55ca17a7a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55ca1842e800 0x55ca17a7a000 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca17a7f800 msgr2=0x55ca17a7b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55ca17a7f800 0x55ca17a7b080 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca17a7ec00 msgr2=0x55ca17a7ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55ca17a7ec00 0x55ca17a7ab00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca1841a800 msgr2=0x55ca1791b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca1841a800 0x55ca1791b700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55ca179a2ff0 tx=0x55ca185b90e0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 msgr2=0x55ca1791a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f2ffc3f4640 1 -- 172.21.15.138:0/4121050682 reap_dead start 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55ca179a3710 tx=0x55ca185b8c60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 shutdown_connections 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 --2- 172.21.15.138:0/4121050682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca1841bc00 0x55ca1791a680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 >> 172.21.15.138:0/4121050682 conn(0x55ca1841b400 msgr2=0x55ca17741800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 shutdown_connections 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.513+0000 7f3000435880 1 -- 172.21.15.138:0/4121050682 wait complete. 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca18598000 msgr2=0x55ca17919080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca18598000 0x55ca17919080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ca185421b0 tx=0x55ca177494a0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 msgr2=0x55ca17918580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55ca184f2ed0 tx=0x55ca17748c60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 shutdown_connections 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55ca18598000 0x55ca17919080 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 --2- 172.21.15.138:0/2434464769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ca17637000 0x55ca17918580 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 >> 172.21.15.138:0/2434464769 conn(0x55ca17636c00 msgr2=0x55ca1773f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 shutdown_connections 2024-04-27T05:21:56.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.514+0000 7f3000435880 1 -- 172.21.15.138:0/2434464769 wait complete. 2024-04-27T05:21:56.527 INFO:teuthology.orchestra.run.smithi138.stdout:[] 2024-04-27T05:21:56.528 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T05:21:56.528 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T05:21:56.528 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T05:21:56.528 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T05:21:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.610+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.610+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.610+0000 7fd769612880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562512941400 0x562512c22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.610+0000 7fd769612880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562512c18600 con 0x562512941000 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.610+0000 7fd769612880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x562512980280 con 0x562512941400 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.610+0000 7fd7652ac640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562512941400 0x562512c22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.611+0000 7fd7652ac640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562512941400 0x562512c22580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44792/0 (socket says 172.21.15.138:44792) 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.611+0000 7fd7652ac640 1 -- 172.21.15.138:0/520461511 learned_addr learned my addr 172.21.15.138:0/520461511 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.611+0000 7fd765aad640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x562512941000 0x562512a48800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45030/0 (socket says 172.21.15.138:45030) 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.611+0000 7fd7652ac640 1 -- 172.21.15.138:0/520461511 >> v1:172.21.15.114:6789/0 conn(0x562512941000 legacy=0x562512a48800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.611+0000 7fd7652ac640 1 -- 172.21.15.138:0/520461511 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562512945380 con 0x562512941400 2024-04-27T05:21:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.612+0000 7fd7652ac640 1 --2- 172.21.15.138:0/520461511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941400 0x562512c22580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x562512a4cc00 tx=0x562512a52de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671d60a9d6f65f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd764aab640 1 -- 172.21.15.138:0/520461511 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562512959ce0 con 0x562512941400 2024-04-27T05:21:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd764aab640 1 -- 172.21.15.138:0/520461511 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562512c19b00 con 0x562512941400 2024-04-27T05:21:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd764aab640 1 -- 172.21.15.138:0/520461511 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562512959e40 con 0x562512941400 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 -- 172.21.15.138:0/520461511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941400 msgr2=0x562512c22580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 --2- 172.21.15.138:0/520461511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941400 0x562512c22580 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x562512a4cc00 tx=0x562512a52de0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 -- 172.21.15.138:0/520461511 shutdown_connections 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 --2- 172.21.15.138:0/520461511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941400 0x562512c22580 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 -- 172.21.15.138:0/520461511 >> 172.21.15.138:0/520461511 conn(0x562512940c00 msgr2=0x562512a48000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 -- 172.21.15.138:0/520461511 shutdown_connections 2024-04-27T05:21:56.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.613+0000 7fd769612880 1 -- 172.21.15.138:0/520461511 wait complete. 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd769612880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd769612880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562512980780 con 0x562512941000 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd765aad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd765aad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44806/0 (socket says 172.21.15.138:44806) 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd765aad640 1 -- 172.21.15.138:0/2476511663 learned_addr learned my addr 172.21.15.138:0/2476511663 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.614+0000 7fd765aad640 1 -- 172.21.15.138:0/2476511663 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562512944ea0 con 0x562512941000 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd765aad640 1 --2- 172.21.15.138:0/2476511663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x5625137fd290 tx=0x562512a52cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6459ee1786d68879 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd7642aa640 1 -- 172.21.15.138:0/2476511663 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625129594a0 con 0x562512941000 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd7642aa640 1 -- 172.21.15.138:0/2476511663 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562512c19e00 con 0x562512941000 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd7642aa640 1 -- 172.21.15.138:0/2476511663 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562512959ce0 con 0x562512941000 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 -- 172.21.15.138:0/2476511663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 msgr2=0x562512c22580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 --2- 172.21.15.138:0/2476511663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x5625137fd290 tx=0x562512a52cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 -- 172.21.15.138:0/2476511663 shutdown_connections 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 --2- 172.21.15.138:0/2476511663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 -- 172.21.15.138:0/2476511663 >> 172.21.15.138:0/2476511663 conn(0x562512940c00 msgr2=0x562512a49000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 -- 172.21.15.138:0/2476511663 shutdown_connections 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 -- 172.21.15.138:0/2476511663 wait complete. 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.615+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd769612880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd769612880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625129808c0 con 0x562512941000 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd765aad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd765aad640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44808/0 (socket says 172.21.15.138:44808) 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd765aad640 1 -- 172.21.15.138:0/3544340388 learned_addr learned my addr 172.21.15.138:0/3544340388 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd765aad640 1 -- 172.21.15.138:0/3544340388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625129456c0 con 0x562512941000 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd765aad640 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5625137fcb40 tx=0x562512a52c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd762aa7640 1 -- 172.21.15.138:0/3544340388 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562512959e40 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd762aa7640 1 -- 172.21.15.138:0/3544340388 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562513812000 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625129451e0 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd762aa7640 1 -- 172.21.15.138:0/3544340388 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625129594a0 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.616+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625129b2ea0 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd762aa7640 1 -- 172.21.15.138:0/3544340388 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56251296b600 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd769612880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd769612880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562512964280 con 0x562513739800 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd762aa7640 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513896000 0x562512c23080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd762aa7640 1 -- 172.21.15.138:0/3544340388 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562512953a40 con 0x562512941000 2024-04-27T05:21:56.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd765aad640 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513896000 0x562512c23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd7662ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd7662ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44822/0 (socket says 172.21.15.138:44822) 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.617+0000 7fd7662ae640 1 -- 172.21.15.138:0/3910971011 learned_addr learned my addr 172.21.15.138:0/3910971011 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd7662ae640 1 -- 172.21.15.138:0/3910971011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625129b2d00 con 0x562513739800 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd765aad640 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513896000 0x562512c23080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x562513844420 tx=0x562512a534a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3910971011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5625137fd560 tx=0x562512a53f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a77edc7855a3c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd7602a2640 1 -- 172.21.15.138:0/3910971011 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562512959e40 con 0x562513739800 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd7602a2640 1 -- 172.21.15.138:0/3910971011 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562513812300 con 0x562513739800 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd7602a2640 1 -- 172.21.15.138:0/3910971011 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625129594a0 con 0x562513739800 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 -- 172.21.15.138:0/3910971011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 msgr2=0x562512c22b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 --2- 172.21.15.138:0/3910971011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x5625137fd560 tx=0x562512a53f80 comp rx=0 tx=0).stop 2024-04-27T05:21:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 -- 172.21.15.138:0/3910971011 shutdown_connections 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 --2- 172.21.15.138:0/3910971011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 -- 172.21.15.138:0/3910971011 >> 172.21.15.138:0/3910971011 conn(0x562512941400 msgr2=0x562512a4b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 -- 172.21.15.138:0/3910971011 shutdown_connections 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.618+0000 7fd769612880 1 -- 172.21.15.138:0/3910971011 wait complete. 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd769612880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd769612880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562512965180 con 0x562513739800 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd7662ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd7662ae640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44824/0 (socket says 172.21.15.138:44824) 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 learned_addr learned my addr 172.21.15.138:0/3695787199 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625129b2b60 con 0x562513739800 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5625137fd980 tx=0x562512a53bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd75ea9f640 1 -- 172.21.15.138:0/3695787199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562512959e40 con 0x562513739800 2024-04-27T05:21:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd75ea9f640 1 -- 172.21.15.138:0/3695787199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562513812480 con 0x562513739800 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd75ea9f640 1 -- 172.21.15.138:0/3695787199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625129594a0 con 0x562513739800 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.619+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625129b29c0 con 0x562513739800 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.620+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625129b2820 con 0x562513739800 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.620+0000 7fd75ea9f640 1 -- 172.21.15.138:0/3695787199 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56251296b8c0 con 0x562513739800 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd75ea9f640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562512cd0000 0x562512c23600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd75ea9f640 1 -- 172.21.15.138:0/3695787199 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562512953c00 con 0x562513739800 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562512cd0000 0x562512c23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513739c00 0x562512c23b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0400 con 0x562513739c00 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513739c00 0x562512c23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562512cd0000 0x562512c23600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x562512a4cc00 tx=0x5625138bee40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513739c00 0x562512c23b80 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 20 realm 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.621+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0800 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0c00 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1000 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512cd1400 0x562512c24100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0400 con 0x562512cd1400 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd765aad640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512cd1400 0x562512c24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.622+0000 7fd765aad640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512cd1400 0x562512c24100 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd769612880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd769612880 20 searching for the correct realm 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0800 con 0x562512cd1400 2024-04-27T05:21:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd765aad640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512cd0800 0x562512c24c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0400 con 0x562512cd0800 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512cd0800 0x562512c24c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.623+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512cd0800 0x562512c24c00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512cd0400 0x562512c25180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512cd0400 0x562512c25180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512cd0400 0x562512c25180 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.624+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0800 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.625+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.626+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0800 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.627+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.628+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0800 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.629+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0800 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.630+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.630+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd1400 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.630+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.630+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0400 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.630+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.630+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562513739c00 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513738800 con 0x562512cd0800 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0c00 con 0x562513739c00 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1000 con 0x562513739c00 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.631+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1800 con 0x562512cd1400 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 zone default found 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 4 Realm: () 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 zonegroup default 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1c00 con 0x562512cd0400 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724000 con 0x562512cd1400 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724400 con 0x562512cd1400 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 started sync module instance, tier type = 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.632+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724800 con 0x562512cd0800 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.633+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.633+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94717228460288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724c00 con 0x562512cd0800 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.633+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513725000 con 0x562513739c00 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.634+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94717228460288] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.635+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.635+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94717228461440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513725400 con 0x562513739c00 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.635+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513725800 con 0x562512cd1400 2024-04-27T05:21:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.636+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.636+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94717228462592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724000 con 0x562512cd1400 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.636+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724400 con 0x562512cd0800 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.636+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94717228461440] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.637+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94717228462592] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.637+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.637+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94717228463744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513725800 con 0x562512cd0800 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.637+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1800 con 0x562513739c00 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.638+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94717228463744] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.638+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.638+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94717228464896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd0c00 con 0x562513739c00 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.638+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1000 con 0x562512cd0400 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.639+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94717228464896] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.639+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.639+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94717228466048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1c00 con 0x562512cd0400 2024-04-27T05:21:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.639+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513896c00 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.640+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94717228466048] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.640+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.640+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94717228467200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724800 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.640+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724c00 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94717228467200] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94717228468352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724400 con 0x562512cd0400 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=0 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=1 2024-04-27T05:21:56.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=2 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=3 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=4 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=5 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.641+0000 7fd769612880 20 add_watcher() i=6 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.642+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94717228468352] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.642+0000 7fd769612880 20 add_watcher() i=7 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.642+0000 7fd769612880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.642+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513725800 con 0x562513739c00 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.642+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.642+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94717228469504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513725000 con 0x562513739c00 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.643+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94717228469504] v31'179 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.643+0000 7fd74ca7b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.643+0000 7fd769612880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.643+0000 7fd769612880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.643+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.643+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd769612880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd769612880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562512964780 con 0x562513725c00 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd7652ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd7652ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44826/0 (socket says 172.21.15.138:44826) 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd7652ac640 1 -- 172.21.15.138:0/2325682554 learned_addr learned my addr 172.21.15.138:0/2325682554 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd7652ac640 1 -- 172.21.15.138:0/2325682554 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625129b2680 con 0x562513725c00 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd7652ac640 1 --2- 172.21.15.138:0/2325682554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5625138c0b10 tx=0x5625138beea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de1f12401b5e32e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.644+0000 7fd74c27a640 1 -- 172.21.15.138:0/2325682554 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562513726580 con 0x562513725c00 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd74c27a640 1 -- 172.21.15.138:0/2325682554 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562513812000 con 0x562513725c00 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd74c27a640 1 -- 172.21.15.138:0/2325682554 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625137266e0 con 0x562513725c00 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 -- 172.21.15.138:0/2325682554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 msgr2=0x562512c24680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 --2- 172.21.15.138:0/2325682554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x5625138c0b10 tx=0x5625138beea0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 -- 172.21.15.138:0/2325682554 shutdown_connections 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 --2- 172.21.15.138:0/2325682554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 -- 172.21.15.138:0/2325682554 >> 172.21.15.138:0/2325682554 conn(0x562513725400 msgr2=0x562512a4b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 -- 172.21.15.138:0/2325682554 shutdown_connections 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 -- 172.21.15.138:0/2325682554 wait complete. 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.645+0000 7fd769612880 1 Processor -- start 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd769612880 1 -- start start 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd769612880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd769612880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562512964500 con 0x562513725c00 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd7652ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd7652ac640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44840/0 (socket says 172.21.15.138:44840) 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 learned_addr learned my addr 172.21.15.138:0/3852491935 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625129b24e0 con 0x562513725c00 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x562512cb3620 tx=0x5625138bed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd74aa77640 1 -- 172.21.15.138:0/3852491935 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562513726840 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd74aa77640 1 -- 172.21.15.138:0/3852491935 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562513812780 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562512944ea0 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd74aa77640 1 -- 172.21.15.138:0/3852491935 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562513726580 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.646+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625129456c0 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd74aa77640 1 -- 172.21.15.138:0/3852491935 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56251296ab00 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd74aa77640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513724800 0x562512c25700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd74aa77640 1 -- 172.21.15.138:0/3852491935 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562512953dc0 con 0x562513725c00 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513724800 0x562512c25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513738800 0x562512d86000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a000 con 0x562513738800 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.647+0000 7fd765aad640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513738800 0x562512d86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.648+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513724800 0x562512c25700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x562512cb2f30 tx=0x5625138bf3e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.648+0000 7fd765aad640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513738800 0x562512d86000 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.649+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'57 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562513738800 2024-04-27T05:21:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.649+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512d8a400 0x562512d86580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.649+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a800 con 0x562512d8a400 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.649+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512d8a400 0x562512d86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.649+0000 7fd7652ac640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512d8a400 0x562512d86580 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.650+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.650+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512d8ac00 0x562512d86b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.650+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b000 con 0x562512d8ac00 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.650+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512d8ac00 0x562512d86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.650+0000 7fd7662ae640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512d8ac00 0x562512d86b00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.651+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.651+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b400 con 0x562513738800 2024-04-27T05:21:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.652+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'180 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562513738800 2024-04-27T05:21:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.652+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512d8b800 0x562512d87080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.652+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8bc00 con 0x562512d8b800 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.652+0000 7fd765aad640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512d8b800 0x562512d87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.653+0000 7fd765aad640 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512d8b800 0x562512d87080 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.653+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'65 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.654+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a000 con 0x562512d8ac00 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.654+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'54 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.654+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a400 con 0x562513738800 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.655+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'32 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562513738800 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.655+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a800 con 0x562512d8a400 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.656+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'45 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.656+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9ac00 con 0x562512d8b800 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.657+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.657+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9b000 con 0x562512d8ac00 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.658+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.658+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9b400 con 0x562512d8b800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.659+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'66 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.659+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9b800 con 0x562512d8a400 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.660+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'27 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.660+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9bc00 con 0x562512d8b800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.661+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.661+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a400 con 0x562512d8a400 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.662+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'62 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.662+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9ac00 con 0x562513738800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.663+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'181 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562513738800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.663+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a000 con 0x562512d8ac00 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.664+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'59 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.664+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b400 con 0x562512d8b800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.665+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'67 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.665+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8bc00 con 0x562512d8ac00 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.666+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'35 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.666+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724000 con 0x562513738800 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.667+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562513738800 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.667+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b000 con 0x562513738800 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.668+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'182 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562513738800 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.668+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a000 con 0x562512d8ac00 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.669+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'60 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.669+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8bc00 con 0x562512d8b800 2024-04-27T05:21:56.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.669+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'39 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.670+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a000 con 0x562512d8ac00 2024-04-27T05:21:56.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.670+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.670+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9b000 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.671+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'41 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.671+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724c00 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.672+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.672+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724400 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.673+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'55 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.673+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b400 con 0x562512d8a400 2024-04-27T05:21:56.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.674+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'32 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.674+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b000 con 0x562512d8ac00 2024-04-27T05:21:56.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.675+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'35 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.675+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8bc00 con 0x562512d8a400 2024-04-27T05:21:56.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.676+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'46 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.676+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724000 con 0x562512d8ac00 2024-04-27T05:21:56.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.677+0000 7fd7662ae640 1 -- 172.21.15.138:0/3852491935 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'61 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512d8ac00 2024-04-27T05:21:56.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.677+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9b400 con 0x562512d8a400 2024-04-27T05:21:56.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.678+0000 7fd7652ac640 1 -- 172.21.15.138:0/3852491935 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'63 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512d8a400 2024-04-27T05:21:56.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.678+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9bc00 con 0x562512d8b800 2024-04-27T05:21:56.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512d8b800 2024-04-27T05:21:56.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd748272640 20 reqs_thread_entry: start 2024-04-27T05:21:56.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd769612880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:56.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd769612880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd74626e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:26 2024 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd745a4c640 20 reqs_thread_entry: start 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd769612880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd769612880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.679+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a800 con 0x562513739c00 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd769612880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd769612880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd769612880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd769612880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b400 con 0x562512cd0800 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.680+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T05:21:56.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.681+0000 7fd74ca7b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.681+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94717228469504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a800 con 0x562513739c00 2024-04-27T05:21:56.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.682+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94717228469504] v31'183 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.682+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94717228460288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9b800 con 0x562512cd0800 2024-04-27T05:21:56.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.683+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94717228460288] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.683+0000 7fd769612880 20 remove_watcher() i=0 2024-04-27T05:21:56.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.683+0000 7fd769612880 2 removed watcher, disabling cache 2024-04-27T05:21:56.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.683+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94717228461440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d9a400 con 0x562513739c00 2024-04-27T05:21:56.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.684+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94717228461440] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.684+0000 7fd769612880 20 remove_watcher() i=1 2024-04-27T05:21:56.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.684+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94717228462592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1c00 con 0x562512cd1400 2024-04-27T05:21:56.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.685+0000 7fd765aad640 1 -- 172.21.15.138:0/3695787199 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94717228462592] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6fc0 con 0x562512cd1400 2024-04-27T05:21:56.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.685+0000 7fd769612880 20 remove_watcher() i=2 2024-04-27T05:21:56.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.685+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94717228463744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512cd1000 con 0x562512cd0800 2024-04-27T05:21:56.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.686+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94717228463744] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce7200 con 0x562512cd0800 2024-04-27T05:21:56.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.686+0000 7fd769612880 20 remove_watcher() i=3 2024-04-27T05:21:56.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.686+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94717228464896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a000 con 0x562513739c00 2024-04-27T05:21:56.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.687+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94717228464896] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562513739c00 2024-04-27T05:21:56.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.687+0000 7fd769612880 20 remove_watcher() i=4 2024-04-27T05:21:56.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.687+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94717228466048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8b000 con 0x562512cd0400 2024-04-27T05:21:56.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.688+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94717228466048] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.688+0000 7fd769612880 20 remove_watcher() i=5 2024-04-27T05:21:56.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.688+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94717228467200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562512d8a800 con 0x562512cd0400 2024-04-27T05:21:56.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.689+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94717228467200] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.689+0000 7fd769612880 20 remove_watcher() i=6 2024-04-27T05:21:56.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.689+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94717228468352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562513724c00 con 0x562512cd0400 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd7662ae640 1 -- 172.21.15.138:0/3695787199 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94717228468352] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562512ce6900 con 0x562512cd0400 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 20 remove_watcher() i=7 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512cd0400 msgr2=0x562512c25180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512cd0400 0x562512c25180 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513739c00 msgr2=0x562512c23b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513739c00 0x562512c23b80 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512cd0800 msgr2=0x562512c24c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512cd0800 0x562512c24c00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512cd1400 msgr2=0x562512c24100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512cd1400 0x562512c24100 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562512cd0000 msgr2=0x562512c23600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562512cd0000 0x562512c23600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x562512a4cc00 tx=0x5625138bee40 comp rx=0 tx=0).stop 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd7652ac640 1 -- 172.21.15.138:0/3695787199 reap_dead start 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 msgr2=0x562512c22b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x5625137fd980 tx=0x562512a53bc0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 shutdown_connections 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 --2- 172.21.15.138:0/3695787199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513739800 0x562512c22b00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 >> 172.21.15.138:0/3695787199 conn(0x562512941400 msgr2=0x562512a4b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 shutdown_connections 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.690+0000 7fd769612880 1 -- 172.21.15.138:0/3695787199 wait complete. 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512d8a400 msgr2=0x562512d86580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562512d8a400 0x562512d86580 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513738800 msgr2=0x562512d86000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562513738800 0x562512d86000 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512d8b800 msgr2=0x562512d87080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562512d8b800 0x562512d87080 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512d8ac00 msgr2=0x562512d86b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562512d8ac00 0x562512d86b00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513724800 msgr2=0x562512c25700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513724800 0x562512c25700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x562512cb2f30 tx=0x5625138bf3e0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 msgr2=0x562512c24680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd765aad640 1 -- 172.21.15.138:0/3852491935 reap_dead start 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x562512cb3620 tx=0x5625138bed20 comp rx=0 tx=0).stop 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 shutdown_connections 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3852491935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562513725c00 0x562512c24680 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 >> 172.21.15.138:0/3852491935 conn(0x562513725400 msgr2=0x562512a4b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 shutdown_connections 2024-04-27T05:21:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3852491935 wait complete. 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513896000 msgr2=0x562512c23080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513896000 0x562512c23080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x562513844420 tx=0x562512a534a0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 msgr2=0x562512c22580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5625137fcb40 tx=0x562512a52c60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 shutdown_connections 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562513896000 0x562512c23080 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 --2- 172.21.15.138:0/3544340388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562512941000 0x562512c22580 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 >> 172.21.15.138:0/3544340388 conn(0x562512940c00 msgr2=0x562512a49000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 shutdown_connections 2024-04-27T05:21:56.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.691+0000 7fd769612880 1 -- 172.21.15.138:0/3544340388 wait complete. 2024-04-27T05:21:56.698 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T05:21:56.706 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-04-27T05:21:56.711 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:56.742 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:56.751 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:56.795 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-27T05:21:56.799 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:56.799 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T05:21:56.799 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T05:21:56.799 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-27T05:21:56.842 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:56.843 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbef40880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbef40880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a434a10600 con 0x55a434739400 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbef40880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a434778280 con 0x55a434739000 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbb943640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbb943640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44846/0 (socket says 172.21.15.138:44846) 2024-04-27T05:21:56.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbb142640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55a434739400 0x55a434840800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45040/0 (socket says 172.21.15.138:45040) 2024-04-27T05:21:56.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.842+0000 7f3dbb943640 1 -- 172.21.15.138:0/994681054 learned_addr learned my addr 172.21.15.138:0/994681054 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.843+0000 7f3dbb943640 1 -- 172.21.15.138:0/994681054 >> v1:172.21.15.114:6789/0 conn(0x55a434739400 legacy=0x55a434840800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.843+0000 7f3dbb943640 1 -- 172.21.15.138:0/994681054 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43473d380 con 0x55a434739000 2024-04-27T05:21:56.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.844+0000 7f3dbb943640 1 --2- 172.21.15.138:0/994681054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55a434844d50 tx=0x55a43484ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c75c34b1cfb8c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.844+0000 7f3dba941640 1 -- 172.21.15.138:0/994681054 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434751ce0 con 0x55a434739000 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.844+0000 7f3dba941640 1 -- 172.21.15.138:0/994681054 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a434a11b00 con 0x55a434739000 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.844+0000 7f3dba941640 1 -- 172.21.15.138:0/994681054 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434751e40 con 0x55a434739000 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 -- 172.21.15.138:0/994681054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 msgr2=0x55a434a1a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 --2- 172.21.15.138:0/994681054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55a434844d50 tx=0x55a43484ade0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 -- 172.21.15.138:0/994681054 shutdown_connections 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 --2- 172.21.15.138:0/994681054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 -- 172.21.15.138:0/994681054 >> 172.21.15.138:0/994681054 conn(0x55a434738c00 msgr2=0x55a434840000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 -- 172.21.15.138:0/994681054 shutdown_connections 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.845+0000 7f3dbef40880 1 -- 172.21.15.138:0/994681054 wait complete. 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbef40880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbef40880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a434778780 con 0x55a434739000 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbb943640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbb943640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44856/0 (socket says 172.21.15.138:44856) 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbb943640 1 -- 172.21.15.138:0/1670215294 learned_addr learned my addr 172.21.15.138:0/1670215294 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbb943640 1 -- 172.21.15.138:0/1670215294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43473cea0 con 0x55a434739000 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dbb943640 1 --2- 172.21.15.138:0/1670215294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55a43553d560 tx=0x55a43484acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c0009f51d44645 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.846+0000 7f3dba140640 1 -- 172.21.15.138:0/1670215294 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4347514a0 con 0x55a434739000 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dba140640 1 -- 172.21.15.138:0/1670215294 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a434a11e00 con 0x55a434739000 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dba140640 1 -- 172.21.15.138:0/1670215294 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434751ce0 con 0x55a434739000 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- 172.21.15.138:0/1670215294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 msgr2=0x55a434a1a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 --2- 172.21.15.138:0/1670215294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55a43553d560 tx=0x55a43484acc0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- 172.21.15.138:0/1670215294 shutdown_connections 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 --2- 172.21.15.138:0/1670215294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- 172.21.15.138:0/1670215294 >> 172.21.15.138:0/1670215294 conn(0x55a434738c00 msgr2=0x55a434841000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- 172.21.15.138:0/1670215294 shutdown_connections 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- 172.21.15.138:0/1670215294 wait complete. 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.847+0000 7f3dbef40880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4347788c0 con 0x55a434739000 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbb943640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbb943640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44858/0 (socket says 172.21.15.138:44858) 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbb943640 1 -- 172.21.15.138:0/3869527015 learned_addr learned my addr 172.21.15.138:0/3869527015 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbb943640 1 -- 172.21.15.138:0/3869527015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43473d6c0 con 0x55a434739000 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbb943640 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x55a4355f52c0 tx=0x55a43484ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3db893d640 1 -- 172.21.15.138:0/3869527015 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434751e40 con 0x55a434739000 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3db893d640 1 -- 172.21.15.138:0/3869527015 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a435518000 con 0x55a434739000 2024-04-27T05:21:56.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3db893d640 1 -- 172.21.15.138:0/3869527015 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4347514a0 con 0x55a434739000 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a43473d1e0 con 0x55a434739000 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.848+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4347aaea0 con 0x55a434739000 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbef40880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbef40880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43475c280 con 0x55a435530c00 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3db893d640 1 -- 172.21.15.138:0/3869527015 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a434762b00 con 0x55a434739000 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbc144640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbc144640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44866/0 (socket says 172.21.15.138:44866) 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbc144640 1 -- 172.21.15.138:0/3387398496 learned_addr learned my addr 172.21.15.138:0/3387398496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3db893d640 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435531c00 0x55a434a1b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3db893d640 1 -- 172.21.15.138:0/3869527015 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55a43474ba40 con 0x55a434739000 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbb943640 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435531c00 0x55a434a1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbc144640 1 -- 172.21.15.138:0/3387398496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4347aad00 con 0x55a435530c00 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbc144640 1 --2- 172.21.15.138:0/3387398496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55a435625800 tx=0x55a43484bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf93c6769f7df92 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3dbb943640 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435531c00 0x55a434a1b080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a435624660 tx=0x55a43484bd40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.849+0000 7f3db6138640 1 -- 172.21.15.138:0/3387398496 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434751e40 con 0x55a435530c00 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3db6138640 1 -- 172.21.15.138:0/3387398496 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a435518600 con 0x55a435530c00 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3db6138640 1 -- 172.21.15.138:0/3387398496 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4347514a0 con 0x55a435530c00 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- 172.21.15.138:0/3387398496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 msgr2=0x55a434a1ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 --2- 172.21.15.138:0/3387398496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55a435625800 tx=0x55a43484bbc0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- 172.21.15.138:0/3387398496 shutdown_connections 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 --2- 172.21.15.138:0/3387398496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- 172.21.15.138:0/3387398496 >> 172.21.15.138:0/3387398496 conn(0x55a434739400 msgr2=0x55a434843000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- 172.21.15.138:0/3387398496 shutdown_connections 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- 172.21.15.138:0/3387398496 wait complete. 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.850+0000 7f3dbef40880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43475d180 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbc144640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbc144640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44870/0 (socket says 172.21.15.138:44870) 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 learned_addr learned my addr 172.21.15.138:0/847346505 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4347aab60 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbc144640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55a4355f5530 tx=0x55a43484b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3db4935640 1 -- 172.21.15.138:0/847346505 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434751e40 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3db4935640 1 -- 172.21.15.138:0/847346505 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a435518780 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a4347aa9c0 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3db4935640 1 -- 172.21.15.138:0/847346505 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4347514a0 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.851+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4347aa820 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3db4935640 1 -- 172.21.15.138:0/847346505 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a434763600 con 0x55a435530c00 2024-04-27T05:21:56.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3db4935640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a434ab6000 0x55a434a1b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3db4935640 1 -- 172.21.15.138:0/847346505 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55a43474bc00 con 0x55a435530c00 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbb142640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a434ab6000 0x55a434a1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a435531000 0x55a434a1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435531400 con 0x55a435531000 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbc144640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a435531000 0x55a434a1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbb142640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a434ab6000 0x55a434a1b600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55a434845380 tx=0x55a4356a8d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.852+0000 7f3dbc144640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a435531000 0x55a434a1bb80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 realm 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435531800 con 0x55a435531000 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7000 con 0x55a435531000 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7400 con 0x55a435531000 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a434ab7800 0x55a434a1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.853+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7c00 con 0x55a434ab7800 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb943640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a434ab7800 0x55a434a1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb943640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a434ab7800 0x55a434a1c100 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbef40880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbef40880 20 searching for the correct realm 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7000 con 0x55a434ab7800 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb943640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a434ab7000 0x55a434a1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7c00 con 0x55a434ab7000 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.854+0000 7f3dbb142640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a434ab7000 0x55a434a1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.855+0000 7f3dbb142640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a434ab7000 0x55a434a1cc00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.855+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.855+0000 7f3dbb142640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a434ab7c00 0x55a434a1d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.855+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.855+0000 7f3dbc144640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a434ab7c00 0x55a434a1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.855+0000 7f3dbc144640 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a434ab7c00 0x55a434a1d180 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.856+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.857+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7000 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.858+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.859+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.860+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7000 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7800 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.861+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7c00 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a435531000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab6c00 con 0x55a434ab7000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7400 con 0x55a435531000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.862+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435531400 con 0x55a435531000 2024-04-27T05:21:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435531800 con 0x55a434ab7800 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 zone default found 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 4 Realm: () 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 zonegroup default 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace000 con 0x55a434ab7c00 2024-04-27T05:21:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace400 con 0x55a434ab7800 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.863+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace800 con 0x55a434ab7800 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.864+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.864+0000 7f3dbef40880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.864+0000 7f3dbef40880 20 started sync module instance, tier type = 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.864+0000 7f3dbef40880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.864+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acec00 con 0x55a434ab7000 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.865+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.865+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94163745974528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acf000 con 0x55a434ab7000 2024-04-27T05:21:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.865+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acf400 con 0x55a435531000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.866+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94163745974528] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.866+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.866+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94163745975680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acf800 con 0x55a435531000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.866+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acfc00 con 0x55a434ab7800 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.867+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94163745975680] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.867+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.867+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94163745976832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace400 con 0x55a434ab7800 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.867+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace800 con 0x55a434ab7000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.868+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94163745976832] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.868+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.868+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94163745977984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acfc00 con 0x55a434ab7000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.868+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace000 con 0x55a435531000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.869+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94163745977984] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.869+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.869+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94163745979136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435531800 con 0x55a435531000 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.869+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435531400 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.870+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.870+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94163745980288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ab7400 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.870+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435697800 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.870+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94163745979136] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.871+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94163745980288] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.871+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.871+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94163745981440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a435697400 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.871+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acec00 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94163745981440] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94163745982592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acf000 con 0x55a434ab7c00 2024-04-27T05:21:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=0 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=1 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=2 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=3 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=4 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=5 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.872+0000 7f3dbef40880 20 add_watcher() i=6 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.873+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94163745982592] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.873+0000 7f3dbef40880 20 add_watcher() i=7 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.873+0000 7f3dbef40880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.873+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434ace800 con 0x55a435531000 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.873+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.873+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94163745983744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434acf400 con 0x55a435531000 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.874+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94163745983744] v31'184 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.874+0000 7f3da2911640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbef40880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbef40880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbef40880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbef40880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43475c780 con 0x55a434ab6c00 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbb142640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbb142640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44884/0 (socket says 172.21.15.138:44884) 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbb142640 1 -- 172.21.15.138:0/2980124053 learned_addr learned my addr 172.21.15.138:0/2980124053 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbb142640 1 -- 172.21.15.138:0/2980124053 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4347aa680 con 0x55a434ab6c00 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.875+0000 7f3dbb142640 1 --2- 172.21.15.138:0/2980124053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55a434a8ec30 tx=0x55a4356a8d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f864d127644797d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3da2110640 1 -- 172.21.15.138:0/2980124053 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434adc580 con 0x55a434ab6c00 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3da2110640 1 -- 172.21.15.138:0/2980124053 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a435518000 con 0x55a434ab6c00 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3da2110640 1 -- 172.21.15.138:0/2980124053 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434adc6e0 con 0x55a434ab6c00 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 -- 172.21.15.138:0/2980124053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 msgr2=0x55a434a1c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 --2- 172.21.15.138:0/2980124053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55a434a8ec30 tx=0x55a4356a8d20 comp rx=0 tx=0).stop 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 -- 172.21.15.138:0/2980124053 shutdown_connections 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 --2- 172.21.15.138:0/2980124053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 -- 172.21.15.138:0/2980124053 >> 172.21.15.138:0/2980124053 conn(0x55a434acf800 msgr2=0x55a434843800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 -- 172.21.15.138:0/2980124053 shutdown_connections 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 -- 172.21.15.138:0/2980124053 wait complete. 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.876+0000 7f3dbef40880 1 Processor -- start 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbef40880 1 -- start start 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbef40880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbef40880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43475c500 con 0x55a434ab6c00 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbb142640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbb142640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44890/0 (socket says 172.21.15.138:44890) 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbb142640 1 -- 172.21.15.138:0/2059428285 learned_addr learned my addr 172.21.15.138:0/2059428285 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbb142640 1 -- 172.21.15.138:0/2059428285 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4347aa4e0 con 0x55a434ab6c00 2024-04-27T05:21:56.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbb142640 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55a434a8f740 tx=0x55a4356a91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3da090d640 1 -- 172.21.15.138:0/2059428285 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434adc840 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3da090d640 1 -- 172.21.15.138:0/2059428285 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a435518a80 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a43473d380 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3da090d640 1 -- 172.21.15.138:0/2059428285 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a434adc580 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.877+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a43473cea0 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.878+0000 7f3da090d640 1 -- 172.21.15.138:0/2059428285 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55a434763b80 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.878+0000 7f3da090d640 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435697800 0x55a434a1d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.878+0000 7f3da090d640 1 -- 172.21.15.138:0/2059428285 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55a43474bdc0 con 0x55a434ab6c00 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.878+0000 7f3dbef40880 5 note: GC not initialized 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.878+0000 7f3dbc144640 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435697800 0x55a434a1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3d9e108640 20 reqs_thread_entry: start 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbc144640 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435697800 0x55a434a1d700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55a434a8ee40 tx=0x55a4356a9020 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:56.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3d9c104640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:27 2024 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3d9b8e2640 20 reqs_thread_entry: start 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434b9c800 con 0x55a434ab7000 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 : miss 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 20 rados->read ofs=0 len=0 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.879+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.4 8:26caafa4:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434b9cc00 con 0x55a434ab7000 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 info.flags=0x17 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 to cache LRU end 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 20 get_obj_state: rctx=0x55a434b90230 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55a43555cde8 s->prefetch_data=0 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:64 3.30 3:0dd8d068:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434b9d000 con 0x55a434ab7800 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(64 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=849b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 262+0+865 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 10 manifest: total_size = 5 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.880+0000 7f3dbef40880 20 get_obj_state: setting s->obj_tag to _iNcsIs4TJDqBkxE6xazyj1Yb135oI0d 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.881+0000 7f3da2911640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.881+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94163745983744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434b9d400 con 0x55a435531000 2024-04-27T05:21:56.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.882+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94163745983744] v31'185 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.882+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94163745974528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434b9d800 con 0x55a434ab7000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.883+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94163745974528] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.883+0000 7f3dbef40880 20 remove_watcher() i=0 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.883+0000 7f3dbef40880 2 removed watcher, disabling cache 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.883+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94163745975680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434b9dc00 con 0x55a435531000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.884+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94163745975680] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.884+0000 7f3dbef40880 20 remove_watcher() i=1 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.884+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94163745976832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434a6a000 con 0x55a434ab7800 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.885+0000 7f3dbb943640 1 -- 172.21.15.138:0/847346505 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94163745976832] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434accfc0 con 0x55a434ab7800 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.885+0000 7f3dbef40880 20 remove_watcher() i=2 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.885+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94163745977984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434a6a400 con 0x55a434ab7000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.886+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94163745977984] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acd200 con 0x55a434ab7000 2024-04-27T05:21:56.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.886+0000 7f3dbef40880 20 remove_watcher() i=3 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.886+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94163745979136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434a6a800 con 0x55a435531000 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.887+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94163745979136] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a435531000 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.887+0000 7f3dbef40880 20 remove_watcher() i=4 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.887+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94163745980288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434a6ac00 con 0x55a434ab7c00 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.888+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94163745980288] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.888+0000 7f3dbef40880 20 remove_watcher() i=5 2024-04-27T05:21:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.888+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94163745981440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434a6b000 con 0x55a434ab7c00 2024-04-27T05:21:56.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.889+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94163745981440] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.889+0000 7f3dbef40880 20 remove_watcher() i=6 2024-04-27T05:21:56.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.889+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94163745982592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55a434a6b400 con 0x55a434ab7c00 2024-04-27T05:21:56.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbc144640 1 -- 172.21.15.138:0/847346505 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94163745982592] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a434acc900 con 0x55a434ab7c00 2024-04-27T05:21:56.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 20 remove_watcher() i=7 2024-04-27T05:21:56.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a434ab7c00 msgr2=0x55a434a1d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55a434ab7c00 0x55a434a1d180 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a435531000 msgr2=0x55a434a1bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55a435531000 0x55a434a1bb80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a434ab7000 msgr2=0x55a434a1cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55a434ab7000 0x55a434a1cc00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a434ab7800 msgr2=0x55a434a1c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55a434ab7800 0x55a434a1c100 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a434ab6000 msgr2=0x55a434a1b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a434ab6000 0x55a434a1b600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55a434845380 tx=0x55a4356a8d80 comp rx=0 tx=0).stop 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 msgr2=0x55a434a1ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/847346505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a435530c00 0x55a434a1ab00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55a4355f5530 tx=0x55a43484b9e0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 reap_dead start 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbb142640 1 -- 172.21.15.138:0/847346505 reap_dead start 2024-04-27T05:21:56.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 shutdown_connections 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 >> 172.21.15.138:0/847346505 conn(0x55a434739400 msgr2=0x55a434843000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 shutdown_connections 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/847346505 wait complete. 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435697800 msgr2=0x55a434a1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435697800 0x55a434a1d700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55a434a8ee40 tx=0x55a4356a9020 comp rx=0 tx=0).stop 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 msgr2=0x55a434a1c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.890+0000 7f3dbef40880 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55a434a8f740 tx=0x55a4356a91a0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 shutdown_connections 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435697800 0x55a434a1d700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 --2- 172.21.15.138:0/2059428285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434ab6c00 0x55a434a1c680 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 >> 172.21.15.138:0/2059428285 conn(0x55a434acf800 msgr2=0x55a434843800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 shutdown_connections 2024-04-27T05:21:56.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/2059428285 wait complete. 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435531c00 msgr2=0x55a434a1b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435531c00 0x55a434a1b080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a435624660 tx=0x55a43484bd40 comp rx=0 tx=0).stop 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 msgr2=0x55a434a1a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x55a4355f52c0 tx=0x55a43484ac60 comp rx=0 tx=0).stop 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 shutdown_connections 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55a435531c00 0x55a434a1b080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 --2- 172.21.15.138:0/3869527015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a434739000 0x55a434a1a580 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 >> 172.21.15.138:0/3869527015 conn(0x55a434738c00 msgr2=0x55a434841000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 shutdown_connections 2024-04-27T05:21:56.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.891+0000 7f3dbef40880 1 -- 172.21.15.138:0/3869527015 wait complete. 2024-04-27T05:21:56.910 INFO:teuthology.orchestra.run.smithi138.stdout:fooFoofooFooFULL_CONTROL 2024-04-27T05:21:56.910 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-04-27T05:21:56.912 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-04-27T05:21:56.916 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:56.917 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T05:21:56.917 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T05:21:56.917 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-27T05:21:56.993 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:56.994 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:56.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:56.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:56.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d7815880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:56.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d7815880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5628b46b4600 con 0x5628b43dd400 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d7815880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5628b441c280 con 0x5628b43dd000 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d3c89640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d3488640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5628b43dd400 0x5628b44e4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45056/0 (socket says 172.21.15.138:45056) 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d3488640 1 -- 172.21.15.138:0/136277530 learned_addr learned my addr 172.21.15.138:0/136277530 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d3c89640 1 --2- 172.21.15.138:0/136277530 >> v2:172.21.15.114:3300/0 conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44904/0 (socket says 172.21.15.138:44904) 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d3c89640 1 -- 172.21.15.138:0/136277530 >> v1:172.21.15.114:6789/0 conn(0x5628b43dd400 legacy=0x5628b44e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d3c89640 1 -- 172.21.15.138:0/136277530 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b43e1380 con 0x5628b43dd000 2024-04-27T05:21:56.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.993+0000 7f15d2c87640 1 -- 172.21.15.138:0/136277530 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815934149 0 0) 0x5628b43e09c0 con 0x5628b43dd400 2024-04-27T05:21:57.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.995+0000 7f15d3c89640 1 --2- 172.21.15.138:0/136277530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x5628b44e8d50 tx=0x5628b44eede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98b3bbd81d49c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.995+0000 7f15d2c87640 1 -- 172.21.15.138:0/136277530 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f5ce0 con 0x5628b43dd000 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.995+0000 7f15d2c87640 1 -- 172.21.15.138:0/136277530 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b46b5b00 con 0x5628b43dd000 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.995+0000 7f15d2c87640 1 -- 172.21.15.138:0/136277530 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f5e40 con 0x5628b43dd000 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 -- 172.21.15.138:0/136277530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 msgr2=0x5628b46be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 --2- 172.21.15.138:0/136277530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x5628b44e8d50 tx=0x5628b44eede0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 -- 172.21.15.138:0/136277530 shutdown_connections 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 --2- 172.21.15.138:0/136277530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 -- 172.21.15.138:0/136277530 >> 172.21.15.138:0/136277530 conn(0x5628b43dcc00 msgr2=0x5628b44e4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 -- 172.21.15.138:0/136277530 shutdown_connections 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 -- 172.21.15.138:0/136277530 wait complete. 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.996+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d7815880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d7815880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b441c780 con 0x5628b43dd000 2024-04-27T05:21:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d3c89640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d3c89640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44920/0 (socket says 172.21.15.138:44920) 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d3c89640 1 -- 172.21.15.138:0/4258139222 learned_addr learned my addr 172.21.15.138:0/4258139222 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d3c89640 1 -- 172.21.15.138:0/4258139222 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b43e0ea0 con 0x5628b43dd000 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d3c89640 1 --2- 172.21.15.138:0/4258139222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x5628b51e3590 tx=0x5628b44eecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320dd07609ab5ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d2486640 1 -- 172.21.15.138:0/4258139222 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f54a0 con 0x5628b43dd000 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d2486640 1 -- 172.21.15.138:0/4258139222 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b46b5e00 con 0x5628b43dd000 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.997+0000 7f15d2486640 1 -- 172.21.15.138:0/4258139222 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f5ce0 con 0x5628b43dd000 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- 172.21.15.138:0/4258139222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 msgr2=0x5628b46be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 --2- 172.21.15.138:0/4258139222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x5628b51e3590 tx=0x5628b44eecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- 172.21.15.138:0/4258139222 shutdown_connections 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 --2- 172.21.15.138:0/4258139222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- 172.21.15.138:0/4258139222 >> 172.21.15.138:0/4258139222 conn(0x5628b43dcc00 msgr2=0x5628b44e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- 172.21.15.138:0/4258139222 shutdown_connections 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- 172.21.15.138:0/4258139222 wait complete. 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d7815880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b441c8c0 con 0x5628b43dd000 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d3c89640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d3c89640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44928/0 (socket says 172.21.15.138:44928) 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d3c89640 1 -- 172.21.15.138:0/1601540791 learned_addr learned my addr 172.21.15.138:0/1601540791 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.998+0000 7f15d3c89640 1 -- 172.21.15.138:0/1601540791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b43e16c0 con 0x5628b43dd000 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d3c89640 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x5628b5298e10 tx=0x5628b44eec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d0c83640 1 -- 172.21.15.138:0/1601540791 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f5e40 con 0x5628b43dd000 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d0c83640 1 -- 172.21.15.138:0/1601540791 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b51bc000 con 0x5628b43dd000 2024-04-27T05:21:57.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628b43e11e0 con 0x5628b43dd000 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d0c83640 1 -- 172.21.15.138:0/1601540791 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f54a0 con 0x5628b43dd000 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b444eea0 con 0x5628b43dd000 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d7815880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:56.999+0000 7f15d7815880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b4400280 con 0x5628b51d4c00 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d0c83640 1 -- 172.21.15.138:0/1601540791 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5628b4406b00 con 0x5628b43dd000 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d448a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d0c83640 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b51d5c00 0x5628b46bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d448a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44940/0 (socket says 172.21.15.138:44940) 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d448a640 1 -- 172.21.15.138:0/3369775899 learned_addr learned my addr 172.21.15.138:0/3369775899 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d0c83640 1 -- 172.21.15.138:0/1601540791 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5628b43efa40 con 0x5628b43dd000 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d3c89640 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b51d5c00 0x5628b46bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d448a640 1 -- 172.21.15.138:0/3369775899 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b444ed00 con 0x5628b51d4c00 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d3c89640 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b51d5c00 0x5628b46bf080 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5628b52dc2d0 tx=0x5628b44efc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15d448a640 1 --2- 172.21.15.138:0/3369775899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5628b5299bc0 tx=0x5628b44eff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6486c478057fe20c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15ce47e640 1 -- 172.21.15.138:0/3369775899 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f5e40 con 0x5628b51d4c00 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15ce47e640 1 -- 172.21.15.138:0/3369775899 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b51bc600 con 0x5628b51d4c00 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.000+0000 7f15ce47e640 1 -- 172.21.15.138:0/3369775899 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f54a0 con 0x5628b51d4c00 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- 172.21.15.138:0/3369775899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 msgr2=0x5628b46beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 --2- 172.21.15.138:0/3369775899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5628b5299bc0 tx=0x5628b44eff80 comp rx=0 tx=0).stop 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- 172.21.15.138:0/3369775899 shutdown_connections 2024-04-27T05:21:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 --2- 172.21.15.138:0/3369775899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- 172.21.15.138:0/3369775899 >> 172.21.15.138:0/3369775899 conn(0x5628b43dd400 msgr2=0x5628b44e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- 172.21.15.138:0/3369775899 shutdown_connections 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- 172.21.15.138:0/3369775899 wait complete. 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d7815880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b4401180 con 0x5628b51d4c00 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d448a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d448a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44956/0 (socket says 172.21.15.138:44956) 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 learned_addr learned my addr 172.21.15.138:0/3483512210 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.001+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b444eb60 con 0x5628b51d4c00 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15d448a640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5628b5299950 tx=0x5628b44ef980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15ccc7b640 1 -- 172.21.15.138:0/3483512210 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f5e40 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15ccc7b640 1 -- 172.21.15.138:0/3483512210 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b51bc780 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15ccc7b640 1 -- 172.21.15.138:0/3483512210 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b43f54a0 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628b444e9c0 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b444e820 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.002+0000 7f15ccc7b640 1 -- 172.21.15.138:0/3483512210 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5628b4407600 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15ccc7b640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b475a000 0x5628b46bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15ccc7b640 1 -- 172.21.15.138:0/3483512210 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5628b43efc00 con 0x5628b51d4c00 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d3488640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b475a000 0x5628b46bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5628b51d5000 0x5628b46bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b51d5400 con 0x5628b51d5000 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d448a640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5628b51d5000 0x5628b46bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d3488640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b475a000 0x5628b46bf600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5628b44e9410 tx=0x5628b534ce40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d448a640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5628b51d5000 0x5628b46bfb80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 20 realm 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.003+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b51d5800 con 0x5628b51d5000 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475b000 con 0x5628b51d5000 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475b400 con 0x5628b51d5000 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5628b475b800 0x5628b46c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475bc00 con 0x5628b475b800 2024-04-27T05:21:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d3c89640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5628b475b800 0x5628b46c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.004+0000 7f15d3c89640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5628b475b800 0x5628b46c0100 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d7815880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d7815880 20 searching for the correct realm 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475b000 con 0x5628b475b800 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d3c89640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5628b475b000 0x5628b46c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475bc00 con 0x5628b475b000 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d3488640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5628b475b000 0x5628b46c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.005+0000 7f15d3488640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5628b475b000 0x5628b46c0c00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d3488640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5628b475bc00 0x5628b46c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d448a640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5628b475bc00 0x5628b46c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d448a640 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5628b475bc00 0x5628b46c1180 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.006+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b000 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.007+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b000 2024-04-27T05:21:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.008+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.009+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b000 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.010+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.011+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b000 2024-04-27T05:21:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b800 2024-04-27T05:21:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475bc00 2024-04-27T05:21:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b51d5000 2024-04-27T05:21:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.012+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475ac00 con 0x5628b475b000 2024-04-27T05:21:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475b400 con 0x5628b51d5000 2024-04-27T05:21:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b51d5400 con 0x5628b51d5000 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b51d5800 con 0x5628b475b800 2024-04-27T05:21:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 zone default found 2024-04-27T05:21:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 4 Realm: () 2024-04-27T05:21:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 zonegroup default 2024-04-27T05:21:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.013+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780000 con 0x5628b475bc00 2024-04-27T05:21:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780400 con 0x5628b475b800 2024-04-27T05:21:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780800 con 0x5628b475b800 2024-04-27T05:21:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 started sync module instance, tier type = 2024-04-27T05:21:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.014+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780c00 con 0x5628b475b000 2024-04-27T05:21:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.015+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.015+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94732825618688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781000 con 0x5628b475b000 2024-04-27T05:21:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.015+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781400 con 0x5628b51d5000 2024-04-27T05:21:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.016+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94732825618688] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.016+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.016+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94732825619840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781800 con 0x5628b51d5000 2024-04-27T05:21:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.016+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781c00 con 0x5628b475b800 2024-04-27T05:21:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.017+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.017+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94732825620992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780400 con 0x5628b475b800 2024-04-27T05:21:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.017+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780800 con 0x5628b475b000 2024-04-27T05:21:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.018+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94732825619840] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.018+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.018+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94732825622144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781c00 con 0x5628b475b000 2024-04-27T05:21:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.018+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780000 con 0x5628b51d5000 2024-04-27T05:21:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.019+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94732825620992] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.019+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.019+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94732825623296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b51d5800 con 0x5628b51d5000 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.019+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b51d5400 con 0x5628b475bc00 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.020+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94732825622144] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.020+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94732825623296] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.021+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.021+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94732825624448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b475b400 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.021+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b533b800 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.021+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94732825624448] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.022+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.022+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94732825625600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b533b400 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.022+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780c00 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94732825625600] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94732825626752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781000 con 0x5628b475bc00 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=0 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=1 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=2 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=3 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=4 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=5 2024-04-27T05:21:57.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.023+0000 7f15d7815880 20 add_watcher() i=6 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.024+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94732825626752] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.024+0000 7f15d7815880 20 add_watcher() i=7 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.024+0000 7f15d7815880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.024+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4780800 con 0x5628b51d5000 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.024+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.024+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94732825627904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b4781400 con 0x5628b51d5000 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.025+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94732825627904] v31'186 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.025+0000 7f15bac57640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.025+0000 7f15d7815880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.025+0000 7f15d7815880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.025+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d7815880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d7815880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b4400780 con 0x5628b475ac00 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d3488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d3488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44970/0 (socket says 172.21.15.138:44970) 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d3488640 1 -- 172.21.15.138:0/3323715874 learned_addr learned my addr 172.21.15.138:0/3323715874 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d3488640 1 -- 172.21.15.138:0/3323715874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b444e680 con 0x5628b475ac00 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.026+0000 7f15d3488640 1 --2- 172.21.15.138:0/3323715874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5628b473f620 tx=0x5628b534cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4233b20d170f6943 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15ba456640 1 -- 172.21.15.138:0/3323715874 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b4782580 con 0x5628b475ac00 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15ba456640 1 -- 172.21.15.138:0/3323715874 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b51bc000 con 0x5628b475ac00 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15ba456640 1 -- 172.21.15.138:0/3323715874 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b47826e0 con 0x5628b475ac00 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 -- 172.21.15.138:0/3323715874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 msgr2=0x5628b46c0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 --2- 172.21.15.138:0/3323715874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5628b473f620 tx=0x5628b534cde0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 -- 172.21.15.138:0/3323715874 shutdown_connections 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 --2- 172.21.15.138:0/3323715874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 -- 172.21.15.138:0/3323715874 >> 172.21.15.138:0/3323715874 conn(0x5628b4781800 msgr2=0x5628b44e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 -- 172.21.15.138:0/3323715874 shutdown_connections 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.027+0000 7f15d7815880 1 -- 172.21.15.138:0/3323715874 wait complete. 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d7815880 1 Processor -- start 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d7815880 1 -- start start 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d7815880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d7815880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b4400500 con 0x5628b475ac00 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d3488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d3488640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44974/0 (socket says 172.21.15.138:44974) 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d3488640 1 -- 172.21.15.138:0/3846685658 learned_addr learned my addr 172.21.15.138:0/3846685658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d3488640 1 -- 172.21.15.138:0/3846685658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b444e4e0 con 0x5628b475ac00 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d3488640 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x5628b473f830 tx=0x5628b534d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15b8c53640 1 -- 172.21.15.138:0/3846685658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b4782840 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15b8c53640 1 -- 172.21.15.138:0/3846685658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628b51bca80 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5628b43e1380 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.028+0000 7f15b8c53640 1 -- 172.21.15.138:0/3846685658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5628b4782580 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.029+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b43e0ea0 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.029+0000 7f15b8c53640 1 -- 172.21.15.138:0/3846685658 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5628b4407b80 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.029+0000 7f15b8c53640 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b533b800 0x5628b46c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.029+0000 7f15b8c53640 1 -- 172.21.15.138:0/3846685658 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5628b43efdc0 con 0x5628b475ac00 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.029+0000 7f15d7815880 5 note: GC not initialized 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.029+0000 7f15d448a640 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b533b800 0x5628b46c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15b644e640 20 reqs_thread_entry: start 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d448a640 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b533b800 0x5628b46c1700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5628b534a720 tx=0x5628b534d320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15b444a640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:27 2024 2024-04-27T05:21:57.076 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15b3c28640 20 reqs_thread_entry: start 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b483e800 con 0x5628b475b000 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 : miss 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.030+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.4 8:26caafa4:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b483ec00 con 0x5628b475b000 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 info.flags=0x17 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 to cache LRU end 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 20 get_obj_state: rctx=0x5628b4860000 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5628b5200de8 s->prefetch_data=0 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:64 3.30 3:0dd8d068:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b483f000 con 0x5628b475b800 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(64 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+927 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 10 manifest: total_size = 5 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.031+0000 7f15d7815880 20 get_obj_state: setting s->obj_tag to _tGMPqiI4k8TgBktfUw08irWJKIbwKUq 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.032+0000 7f15bac57640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.032+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94732825627904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b483f400 con 0x5628b51d5000 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.033+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94732825627904] v31'187 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.033+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94732825618688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b483f800 con 0x5628b475b000 2024-04-27T05:21:57.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.034+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94732825618688] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.034+0000 7f15d7815880 20 remove_watcher() i=0 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.034+0000 7f15d7815880 2 removed watcher, disabling cache 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.034+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94732825619840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b483fc00 con 0x5628b51d5000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.035+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94732825619840] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.035+0000 7f15d7815880 20 remove_watcher() i=1 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.035+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94732825620992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b534e000 con 0x5628b475b800 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.035+0000 7f15d3c89640 1 -- 172.21.15.138:0/3483512210 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94732825620992] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776d80 con 0x5628b475b800 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.036+0000 7f15d7815880 20 remove_watcher() i=2 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.036+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94732825622144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b534e400 con 0x5628b475b000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.036+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94732825622144] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b4776fc0 con 0x5628b475b000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.036+0000 7f15d7815880 20 remove_watcher() i=3 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.036+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94732825623296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b534e800 con 0x5628b51d5000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.037+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94732825623296] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b51d5000 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.037+0000 7f15d7815880 20 remove_watcher() i=4 2024-04-27T05:21:57.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.037+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94732825624448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b534ec00 con 0x5628b475bc00 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.038+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94732825624448] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.038+0000 7f15d7815880 20 remove_watcher() i=5 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.038+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94732825625600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b534f000 con 0x5628b475bc00 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.039+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94732825625600] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.039+0000 7f15d7815880 20 remove_watcher() i=6 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.039+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94732825626752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5628b534f400 con 0x5628b475bc00 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d448a640 1 -- 172.21.15.138:0/3483512210 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94732825626752] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5628b47766c0 con 0x5628b475bc00 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 20 remove_watcher() i=7 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5628b475bc00 msgr2=0x5628b46c1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5628b475bc00 0x5628b46c1180 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5628b51d5000 msgr2=0x5628b46bfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5628b51d5000 0x5628b46bfb80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5628b475b000 msgr2=0x5628b46c0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5628b475b000 0x5628b46c0c00 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5628b475b800 msgr2=0x5628b46c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5628b475b800 0x5628b46c0100 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b475a000 msgr2=0x5628b46bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b475a000 0x5628b46bf600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5628b44e9410 tx=0x5628b534ce40 comp rx=0 tx=0).stop 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 msgr2=0x5628b46beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 --2- 172.21.15.138:0/3483512210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b51d4c00 0x5628b46beb00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5628b5299950 tx=0x5628b44ef980 comp rx=0 tx=0).stop 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 reap_dead start 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d3488640 1 -- 172.21.15.138:0/3483512210 reap_dead start 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 shutdown_connections 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 >> 172.21.15.138:0/3483512210 conn(0x5628b43dd400 msgr2=0x5628b44e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 shutdown_connections 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.040+0000 7f15d7815880 1 -- 172.21.15.138:0/3483512210 wait complete. 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b533b800 msgr2=0x5628b46c1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b533b800 0x5628b46c1700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5628b534a720 tx=0x5628b534d320 comp rx=0 tx=0).stop 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 msgr2=0x5628b46c0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x5628b473f830 tx=0x5628b534d560 comp rx=0 tx=0).stop 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 shutdown_connections 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b533b800 0x5628b46c1700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/3846685658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b475ac00 0x5628b46c0680 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 >> 172.21.15.138:0/3846685658 conn(0x5628b4781800 msgr2=0x5628b44e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 shutdown_connections 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/3846685658 wait complete. 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b51d5c00 msgr2=0x5628b46bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b51d5c00 0x5628b46bf080 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5628b52dc2d0 tx=0x5628b44efc20 comp rx=0 tx=0).stop 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 msgr2=0x5628b46be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x5628b5298e10 tx=0x5628b44eec60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 shutdown_connections 2024-04-27T05:21:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5628b51d5c00 0x5628b46bf080 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 --2- 172.21.15.138:0/1601540791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628b43dd000 0x5628b46be580 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 >> 172.21.15.138:0/1601540791 conn(0x5628b43dcc00 msgr2=0x5628b44e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 shutdown_connections 2024-04-27T05:21:57.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.041+0000 7f15d7815880 1 -- 172.21.15.138:0/1601540791 wait complete. 2024-04-27T05:21:57.084 INFO:teuthology.orchestra.run.smithi138.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-27T05:21:57.084 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-27T05:21:57.086 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-04-27T05:21:57.089 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:57.147 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-27T05:21:57.147 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-27T05:21:57.147 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T05:21:57.147 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-27T05:21:57.147 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-27T05:21:57.147 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-27T05:21:57.148 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-04-27T05:21:57.191 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:57.191 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.190+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.190+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.190+0000 7f389341f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x561965535000 0x561965816580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.190+0000 7f389341f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56196580c600 con 0x561965535400 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.190+0000 7f389341f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x561965574280 con 0x561965535000 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.191+0000 7f388f196640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x561965535400 0x56196563c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45066/0 (socket says 172.21.15.138:45066) 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.191+0000 7f388f997640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x561965535000 0x561965816580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.191+0000 7f388f196640 1 -- 172.21.15.138:0/2342023535 learned_addr learned my addr 172.21.15.138:0/2342023535 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.191+0000 7f388f997640 1 -- 172.21.15.138:0/2342023535 >> v1:172.21.15.114:6789/0 conn(0x561965535400 legacy=0x56196563c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.191+0000 7f388f997640 1 -- 172.21.15.138:0/2342023535 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561965539380 con 0x561965535000 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f388f997640 1 --2- 172.21.15.138:0/2342023535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x561965640d50 tx=0x561965646de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36caf447b1ac3051 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f388e995640 1 -- 172.21.15.138:0/2342023535 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554dce0 con 0x561965535000 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f388e995640 1 -- 172.21.15.138:0/2342023535 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196580db00 con 0x561965535000 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f388e995640 1 -- 172.21.15.138:0/2342023535 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554de40 con 0x561965535000 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 -- 172.21.15.138:0/2342023535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 msgr2=0x561965816580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 --2- 172.21.15.138:0/2342023535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x561965640d50 tx=0x561965646de0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 -- 172.21.15.138:0/2342023535 shutdown_connections 2024-04-27T05:21:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 --2- 172.21.15.138:0/2342023535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 -- 172.21.15.138:0/2342023535 >> 172.21.15.138:0/2342023535 conn(0x561965534c00 msgr2=0x56196563c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 -- 172.21.15.138:0/2342023535 shutdown_connections 2024-04-27T05:21:57.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.193+0000 7f389341f880 1 -- 172.21.15.138:0/2342023535 wait complete. 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f389341f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f389341f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965574780 con 0x561965535000 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f388f997640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f388f997640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:44994/0 (socket says 172.21.15.138:44994) 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.194+0000 7f388f997640 1 -- 172.21.15.138:0/993830572 learned_addr learned my addr 172.21.15.138:0/993830572 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f388f997640 1 -- 172.21.15.138:0/993830572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561965538ea0 con 0x561965535000 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f388f997640 1 --2- 172.21.15.138:0/993830572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x561966335560 tx=0x561965646cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44a579b8855a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f388e194640 1 -- 172.21.15.138:0/993830572 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554d4a0 con 0x561965535000 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f388e194640 1 -- 172.21.15.138:0/993830572 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196580de00 con 0x561965535000 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f388e194640 1 -- 172.21.15.138:0/993830572 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554dce0 con 0x561965535000 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 -- 172.21.15.138:0/993830572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 msgr2=0x561965816580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 --2- 172.21.15.138:0/993830572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x561966335560 tx=0x561965646cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 -- 172.21.15.138:0/993830572 shutdown_connections 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 --2- 172.21.15.138:0/993830572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 -- 172.21.15.138:0/993830572 >> 172.21.15.138:0/993830572 conn(0x561965534c00 msgr2=0x56196563d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 -- 172.21.15.138:0/993830572 shutdown_connections 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.195+0000 7f389341f880 1 -- 172.21.15.138:0/993830572 wait complete. 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f389341f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f389341f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619655748c0 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388f997640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388f997640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45000/0 (socket says 172.21.15.138:45000) 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388f997640 1 -- 172.21.15.138:0/111934428 learned_addr learned my addr 172.21.15.138:0/111934428 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388f997640 1 -- 172.21.15.138:0/111934428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619655396c0 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388f997640 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x5619663e4de0 tx=0x561965646c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388c991640 1 -- 172.21.15.138:0/111934428 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554de40 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388c991640 1 -- 172.21.15.138:0/111934428 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196631a000 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619655391e0 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f388c991640 1 -- 172.21.15.138:0/111934428 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554d4a0 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.196+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619655a6ea0 con 0x561965535000 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f389341f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f389341f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965558280 con 0x561966328c00 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f388c991640 1 -- 172.21.15.138:0/111934428 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56196555eb00 con 0x561965535000 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f388c991640 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966329c00 0x561965817080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f388c991640 1 -- 172.21.15.138:0/111934428 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x561965547a40 con 0x561965535000 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f3890198640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f3890198640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45002/0 (socket says 172.21.15.138:45002) 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.197+0000 7f3890198640 1 -- 172.21.15.138:0/4198721886 learned_addr learned my addr 172.21.15.138:0/4198721886 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f388f997640 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966329c00 0x561965817080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f3890198640 1 -- 172.21.15.138:0/4198721886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619655a6d00 con 0x561966328c00 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f388f997640 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966329c00 0x561965817080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5619664302a0 tx=0x561965647c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f3890198640 1 --2- 172.21.15.138:0/4198721886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5619663e5d10 tx=0x561965647f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475ed166a6914aef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f388a18c640 1 -- 172.21.15.138:0/4198721886 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554de40 con 0x561966328c00 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f388a18c640 1 -- 172.21.15.138:0/4198721886 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196631a600 con 0x561966328c00 2024-04-27T05:21:57.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f388a18c640 1 -- 172.21.15.138:0/4198721886 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554d4a0 con 0x561966328c00 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f389341f880 1 -- 172.21.15.138:0/4198721886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 msgr2=0x561965816b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f389341f880 1 --2- 172.21.15.138:0/4198721886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5619663e5d10 tx=0x561965647f80 comp rx=0 tx=0).stop 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f389341f880 1 -- 172.21.15.138:0/4198721886 shutdown_connections 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f389341f880 1 --2- 172.21.15.138:0/4198721886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.198+0000 7f389341f880 1 -- 172.21.15.138:0/4198721886 >> 172.21.15.138:0/4198721886 conn(0x561965535400 msgr2=0x56196563f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f389341f880 1 -- 172.21.15.138:0/4198721886 shutdown_connections 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f389341f880 1 -- 172.21.15.138:0/4198721886 wait complete. 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f389341f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f389341f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965559180 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f3890198640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f3890198640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45010/0 (socket says 172.21.15.138:45010) 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 learned_addr learned my addr 172.21.15.138:0/454303614 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619655a6b60 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.199+0000 7f3890198640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x5619663e5980 tx=0x561965647140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f3888989640 1 -- 172.21.15.138:0/454303614 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554de40 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f3888989640 1 -- 172.21.15.138:0/454303614 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196631a780 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f3888989640 1 -- 172.21.15.138:0/454303614 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56196554d4a0 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619655a69c0 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619655a6820 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f3888989640 1 -- 172.21.15.138:0/454303614 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56196555f600 con 0x561966328c00 2024-04-27T05:21:57.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f3888989640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619658ae000 0x561965817600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.200+0000 7f3888989640 1 -- 172.21.15.138:0/454303614 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x561965547c00 con 0x561966328c00 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f388f196640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619658ae000 0x561965817600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966329000 0x561965817b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x561966329000 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f3890198640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966329000 0x561965817b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f388f196640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619658ae000 0x561965817600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x561966431260 tx=0x56196648ee40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f3890198640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966329000 0x561965817b80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 20 realm 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329800 con 0x561966329000 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.201+0000 7f389341f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658af000 con 0x561966329000 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658af400 con 0x561966329000 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619658af800 0x561965818100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658afc00 con 0x5619658af800 2024-04-27T05:21:57.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f388f997640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619658af800 0x561965818100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.202+0000 7f388f997640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619658af800 0x561965818100 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f389341f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f389341f880 20 searching for the correct realm 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658af000 con 0x5619658af800 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f997640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619658af000 0x561965818c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658afc00 con 0x5619658af000 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f196640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619658af000 0x561965818c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f196640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619658af000 0x561965818c00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.203+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.204+0000 7f388f196640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619658afc00 0x561965819180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.204+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.204+0000 7f3890198640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619658afc00 0x561965819180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.204+0000 7f3890198640 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619658afc00 0x561965819180 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.204+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.204+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af000 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.205+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.206+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af000 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.207+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.208+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af000 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.209+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af000 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af800 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.210+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658afc00 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x561966329000 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658aec00 con 0x5619658af000 2024-04-27T05:21:57.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658af400 con 0x561966329000 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x561966329000 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.211+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329800 con 0x5619658af800 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 zone default found 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 4 Realm: () 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 zonegroup default 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6000 con 0x5619658afc00 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6400 con 0x5619658af800 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.212+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6800 con 0x5619658af800 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.213+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.213+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.213+0000 7f389341f880 20 started sync module instance, tier type = 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.213+0000 7f389341f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.213+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6c00 con 0x5619658af000 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.214+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.214+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94667077118208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7000 con 0x5619658af000 2024-04-27T05:21:57.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.214+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7400 con 0x561966329000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.215+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94667077118208] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.215+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.215+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94667077119360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7800 con 0x561966329000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.215+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7c00 con 0x5619658af800 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.216+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94667077119360] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.216+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.216+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94667077120512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6400 con 0x5619658af800 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.216+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6800 con 0x5619658af000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.217+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94667077120512] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.217+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.217+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94667077121664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7c00 con 0x5619658af000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.217+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6000 con 0x561966329000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.218+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94667077121664] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.218+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.218+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94667077122816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329800 con 0x561966329000 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.218+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x5619658afc00 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.219+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.219+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94667077123968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658af400 con 0x5619658afc00 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.219+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94667077122816] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.219+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966490c00 con 0x5619658afc00 2024-04-27T05:21:57.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.220+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94667077123968] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.220+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.220+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94667077125120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966490800 con 0x5619658afc00 2024-04-27T05:21:57.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.220+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6c00 con 0x5619658afc00 2024-04-27T05:21:57.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94667077125120] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94667077126272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7000 con 0x5619658afc00 2024-04-27T05:21:57.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=0 2024-04-27T05:21:57.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=1 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=2 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=3 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=4 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=5 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.222+0000 7f389341f880 20 add_watcher() i=6 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.223+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94667077126272] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.223+0000 7f389341f880 20 add_watcher() i=7 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.223+0000 7f389341f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.223+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6800 con 0x561966329000 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.223+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.223+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94667077127424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7400 con 0x561966329000 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.224+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94667077127424] v31'188 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.224+0000 7f3876965640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f389341f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f389341f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f389341f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f389341f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965558780 con 0x5619658aec00 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f388f196640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f388f196640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45022/0 (socket says 172.21.15.138:45022) 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f388f196640 1 -- 172.21.15.138:0/256897816 learned_addr learned my addr 172.21.15.138:0/256897816 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f388f196640 1 -- 172.21.15.138:0/256897816 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619655a6680 con 0x5619658aec00 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.225+0000 7f388f196640 1 --2- 172.21.15.138:0/256897816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x56196588c900 tx=0x56196648ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f70dbc557170016 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f3876164640 1 -- 172.21.15.138:0/256897816 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619658d8580 con 0x5619658aec00 2024-04-27T05:21:57.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f3876164640 1 -- 172.21.15.138:0/256897816 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196631a000 con 0x5619658aec00 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f3876164640 1 -- 172.21.15.138:0/256897816 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619658d86e0 con 0x5619658aec00 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 -- 172.21.15.138:0/256897816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 msgr2=0x561965818680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 --2- 172.21.15.138:0/256897816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x56196588c900 tx=0x56196648ede0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 -- 172.21.15.138:0/256897816 shutdown_connections 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 --2- 172.21.15.138:0/256897816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 -- 172.21.15.138:0/256897816 >> 172.21.15.138:0/256897816 conn(0x5619658d7800 msgr2=0x56196563f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 -- 172.21.15.138:0/256897816 shutdown_connections 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.226+0000 7f389341f880 1 -- 172.21.15.138:0/256897816 wait complete. 2024-04-27T05:21:57.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f389341f880 1 Processor -- start 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f389341f880 1 -- start start 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f389341f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f389341f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965558500 con 0x5619658aec00 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f388f196640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f388f196640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45026/0 (socket says 172.21.15.138:45026) 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 learned_addr learned my addr 172.21.15.138:0/492702467 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619655a64e0 con 0x5619658aec00 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f388f196640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x56196588d890 tx=0x56196648f560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f3874961640 1 -- 172.21.15.138:0/492702467 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619658d8840 con 0x5619658aec00 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f3874961640 1 -- 172.21.15.138:0/492702467 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56196631aa80 con 0x5619658aec00 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561965539380 con 0x5619658aec00 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.227+0000 7f3874961640 1 -- 172.21.15.138:0/492702467 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619658d8580 con 0x5619658aec00 2024-04-27T05:21:57.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561965538ea0 con 0x5619658aec00 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f3874961640 1 -- 172.21.15.138:0/492702467 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56196555fb80 con 0x5619658aec00 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f3874961640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966490c00 0x561965819700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f3874961640 1 -- 172.21.15.138:0/492702467 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x561965547dc0 con 0x5619658aec00 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f3890198640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966490c00 0x561965819700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966490800 0x561965978000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.228+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c000 con 0x561966490800 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.229+0000 7f388f997640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966490800 0x561965978000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.229+0000 7f3890198640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966490c00 0x561965819700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x56196588d200 tx=0x56196648f320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.229+0000 7f388f997640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966490800 0x561965978000 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.230+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'58 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x561966490800 2024-04-27T05:21:57.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.230+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56196597c400 0x561965978580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.230+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c800 con 0x56196597c400 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.230+0000 7f388f196640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56196597c400 0x561965978580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.230+0000 7f388f196640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56196597c400 0x561965978580 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.231+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.231+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56196597cc00 0x561965978b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.231+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d000 con 0x56196597cc00 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.231+0000 7f3890198640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56196597cc00 0x561965978b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.231+0000 7f3890198640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56196597cc00 0x561965978b00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.232+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.232+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x561966490800 2024-04-27T05:21:57.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.233+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'189 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x561966490800 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.233+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56196597d800 0x561965979080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.233+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597dc00 con 0x56196597d800 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.233+0000 7f388f997640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56196597d800 0x561965979080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.234+0000 7f388f997640 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56196597d800 0x561965979080 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.234+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'68 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.234+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994000 con 0x56196597cc00 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.235+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'56 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.235+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994400 con 0x561966490800 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.236+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'33 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x561966490800 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.236+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994800 con 0x56196597c400 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.237+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'47 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.237+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994c00 con 0x56196597d800 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.238+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.238+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995000 con 0x56196597cc00 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.239+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.239+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995400 con 0x56196597d800 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.240+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'69 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.240+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995800 con 0x56196597c400 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.241+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'29 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.241+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995c00 con 0x56196597d800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.242+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.242+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994400 con 0x56196597c400 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.243+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'64 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.243+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994c00 con 0x561966490800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.244+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'190 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x561966490800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.244+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995400 con 0x56196597cc00 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.245+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'65 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.245+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c000 con 0x56196597d800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.246+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'70 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.246+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x56196597cc00 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.247+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'36 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.247+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597dc00 con 0x561966490800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.248+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x561966490800 2024-04-27T05:21:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.248+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6400 con 0x561966490800 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.249+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'191 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x561966490800 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.249+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994000 con 0x56196597cc00 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.250+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'66 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.250+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995000 con 0x56196597d800 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.251+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'40 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.251+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995400 con 0x56196597cc00 2024-04-27T05:21:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.252+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.252+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994800 con 0x56196597cc00 2024-04-27T05:21:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.253+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'42 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.253+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965995800 con 0x56196597cc00 2024-04-27T05:21:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.254+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.254+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965994400 con 0x56196597cc00 2024-04-27T05:21:57.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.255+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'57 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.255+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d000 con 0x56196597c400 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.256+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'33 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.256+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x56196597cc00 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.257+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'36 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.257+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c800 con 0x56196597c400 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.258+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'48 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.258+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x56196597cc00 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.259+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'67 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.259+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6c00 con 0x56196597c400 2024-04-27T05:21:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.260+0000 7f388f196640 1 -- 172.21.15.138:0/492702467 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'65 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bd200 con 0x56196597c400 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.260+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7000 con 0x56196597d800 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x56196597d800 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f387215c640 20 reqs_thread_entry: start 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f389341f880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f389341f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f3870158640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:27 2024 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f386f936640 20 reqs_thread_entry: start 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f389341f880 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.261+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c000 con 0x5619658af000 2024-04-27T05:21:57.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 : miss 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.4 8:26caafa4:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c400 con 0x5619658af000 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(63 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 info.flags=0x17 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 to cache LRU end 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 4.9 4:928c95c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c800 con 0x561966329000 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(64 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.262+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 4.8 4:11b174e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0cc00 con 0x5619658af800 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.263+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.263+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 4.2f 4:f5dc11c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d000 con 0x561966329000 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.263+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 [call out=425b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.263+0000 7f389341f880 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.263+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 4.33 4:ce65ea85:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d400 con 0x5619658afc00 2024-04-27T05:21:57.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.264+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.264+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 4.13 4:ca212683:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d800 con 0x5619658af000 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.264+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.264+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 4.37 4:ecf90668:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0dc00 con 0x5619658afc00 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.264+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.264+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 4.26 4:66f6476b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0cc00 con 0x5619658afc00 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 [call out=425b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f389341f880 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 4.3b 4:dcdfdbe5:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c800 con 0x5619658af800 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.12 4:4933ce3f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x561966329000 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 [call out=424b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+424 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f389341f880 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-04-27T05:21:57.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.265+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.27 4:e5072d9d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c000 con 0x561966329000 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597dc00 con 0x561966329000 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [call out=698b] v0'0 uv11 ondisk = 0) v8 ==== 195+0+698 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 3.1d 3:b979a05a:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597dc00 con 0x5619658af800 2024-04-27T05:21:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(75 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 10 manifest: total_size = 6 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.9077442478998163521 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 20 prepare_atomic_modification: state is not atomic. state=0x561965a121e8 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.266+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:76 4.2f 4:f5dc11c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x561966329000 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.267+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(76 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 [stat,call,call] v31'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.267+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:77 3.1d 3:b979a05a:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6400 con 0x5619658af800 2024-04-27T05:21:57.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.268+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(77 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_9c8454ddf7aa50116496bac348d7550d [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.268+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 4.2f 4:f5dc11c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16000 con 0x561966329000 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.268+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.268+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 3.33 3:cef3d5c1:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16400 con 0x5619658afc00 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(79 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=877b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f389341f880 10 manifest: total_size = 3 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f389341f880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.1808060887878381056 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f389341f880 20 prepare_atomic_modification: state is not atomic. state=0x561965a121e8 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f389341f880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.269+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:80 4.26 4:66f6476b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16800 con 0x5619658afc00 2024-04-27T05:21:57.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.270+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(78 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 [stat,call,call] v31'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.270+0000 7f388a18c640 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.270+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(80 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 [stat,call,call] v31'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.270+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 3.33 3:cef3d5c1:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16c00 con 0x5619658afc00 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(81 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_b1b9a972ccd8c962a473909b97007eb4 [call] v31'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 4.26 4:66f6476b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17400 con 0x5619658afc00 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:83 3.1c 3:3a48b2ac:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17800 con 0x561966329000 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(83 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=875b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+891 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 10 manifest: total_size = 4 2024-04-27T05:21:57.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.152401505742361614 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 20 prepare_atomic_modification: state is not atomic. state=0x561965a121e8 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.271+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:84 4.12 4:4933ce3f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17c00 con 0x561966329000 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.272+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(82 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 [stat,call,call] v31'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.272+0000 7f388998b640 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.272+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(84 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 [stat,call,call] v31'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.272+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:85 3.1c 3:3a48b2ac:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16000 con 0x561966329000 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.273+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(85 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.273+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 4.12 4:4933ce3f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c000 con 0x561966329000 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.273+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.273+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:87 3.3e 3:7c784ea6:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x5619658afc00 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(87 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f389341f880 10 manifest: total_size = 5 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f389341f880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.11106502582673291630 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f389341f880 20 prepare_atomic_modification: state is not atomic. state=0x561965a121e8 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f389341f880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x561966329000 2024-04-27T05:21:57.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(86 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 [stat,call,call] v31'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.274+0000 7f388a18c640 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.275+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(88 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [stat,call,call] v31'12 uv12 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.275+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:89 3.3e 3:7c784ea6:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d000 con 0x5619658afc00 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(89 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_24d27c169c2c881eb09a065116f2aa5c [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:90 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0dc00 con 0x561966329000 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:91 3.30 3:0dd8d068:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0cc00 con 0x5619658af800 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(91 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+927 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 10 manifest: total_size = 5 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 20 get_obj_state: setting s->obj_tag to _tGMPqiI4k8TgBktfUw08irWJKIbwKUq 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 20 get_obj_state: rctx=0x56196647d810 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x561965a121e8 s->prefetch_data=0 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 20 prepare_atomic_modification: state is not atomic. state=0x561965a121e8 2024-04-27T05:21:57.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.276+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:92 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d000 con 0x561966329000 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.277+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(90 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [stat,call,call] v31'13 uv13 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.277+0000 7f388998b640 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.277+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(92 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [stat,call,call] v31'14 uv14 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.277+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:93 3.30 3:0dd8d068:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x5619658af800 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(93 2f19b8ff-2608-4d86-8f1c-430623205476.4231.5_bb3aec0fdcdbc2974890f805c585d432 [call] v31'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x561966329000 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 20 list_objects_ordered: starting attempt 1 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:95 4.9 4:928c95c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16800 con 0x561966329000 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:96 4.8 4:11b174e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16c00 con 0x5619658af800 2024-04-27T05:21:57.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:97 4.2f 4:f5dc11c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17400 con 0x561966329000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:98 4.33 4:ce65ea85:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17800 con 0x5619658afc00 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:99 4.13 4:ca212683:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17c00 con 0x5619658af000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 4.37 4:ecf90668:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16000 con 0x5619658afc00 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 4.26 4:66f6476b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d400 con 0x5619658afc00 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:102 4.3b 4:dcdfdbe5:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17000 con 0x5619658af800 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.278+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:103 4.12 4:4933ce3f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17c00 con 0x561966329000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:104 4.27 4:e5072d9d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c000 con 0x561966329000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:105 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c400 con 0x561966329000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(95 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(99 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(96 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(102 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 36 ==== osd_op_reply(97 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 37 ==== osd_op_reply(103 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 38 ==== osd_op_reply(104 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(100 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(98 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(101 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 39 ==== osd_op_reply(94 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [stat,call,call] v31'15 uv15 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.279+0000 7f388a18c640 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 40 ==== osd_op_reply(105 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [call out=124b] v0'0 uv15 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T05:21:57.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.5])"; uploads.size()=0; is_truncated=0 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.5]) 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x17, cached=0x11) 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d800 con 0x5619658af000 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(106 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv33 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 lifecycle: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:57.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.280+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:107 8.4 8:26caafa4:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d000 con 0x5619658af000 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(107 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [delete,create,call,call,writefull 0~352,setxattr (123)] v31'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f389341f880 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 info.flags=0x17 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f389341f880 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 to cache LRU end 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f389341f880 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f389341f880 10 lifecycle: distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5, ofs0, ns] 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [notify cookie 94667077130880 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0dc00 con 0x5619658af800 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== watch-notify(notify (1) cookie 94667077120512 notify 133143986192 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x561966438d00 con 0x5619658af800 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388a18c640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94667077120512 notifier 4709 bl.length()=725 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388a18c640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 info.flags=0x17 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(108 notify.2 [notify cookie 94667077130880 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388a18c640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 to cache LRU end 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388a18c640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:21:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.281+0000 7f388a18c640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:109 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a22000 con 0x5619658af800 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(109 notify.2 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== watch-notify(notify_complete (2) cookie 94667077130880 notify 133143986192 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561966438ea0 con 0x5619658af800 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f389341f880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.5]): orig_sources= new_sources= 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f389341f880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.5]): potential sources added=[] removed=[] 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f389341f880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.5]): orig_dests= new_dests= 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f389341f880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.5]): potential dests added=[] removed=[] 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.282+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:33 6.7 6:e31d3d5f:lc::lc.28:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d000 con 0x56196597cc00 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.283+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(33 lc.28 [call] v31'68 uv68 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.283+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 6.7 6:e31d3d5f:lc::lc.28:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d400 con 0x56196597cc00 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.284+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(34 lc.28 [call] v31'69 uv69 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.284+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:35 6.7 6:e31d3d5f:lc::lc.28:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c000 con 0x56196597cc00 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/492702467 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(35 lc.28 [call] v31'70 uv70 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x5619658bc900 con 0x56196597cc00 2024-04-27T05:21:57.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:110 4.9 4:928c95c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c400 con 0x561966329000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:111 4.8 4:11b174e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16400 con 0x5619658af800 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:112 4.2f 4:f5dc11c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16800 con 0x561966329000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:113 4.33 4:ce65ea85:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17400 con 0x5619658afc00 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:114 4.13 4:ca212683:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17800 con 0x5619658af000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:115 4.37 4:ecf90668:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16000 con 0x5619658afc00 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:116 4.26 4:66f6476b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17c00 con 0x5619658afc00 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:117 4.3b 4:dcdfdbe5:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c000 con 0x5619658af800 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:118 4.12 4:4933ce3f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x561966329000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:119 4.27 4:e5072d9d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7000 con 0x561966329000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:120 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a22000 con 0x561966329000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(111 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(115 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(114 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== osd_op_reply(113 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== osd_op_reply(116 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 29 ==== osd_op_reply(117 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 41 ==== osd_op_reply(110 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 42 ==== osd_op_reply(112 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 43 ==== osd_op_reply(118 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 44 ==== osd_op_reply(119 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 45 ==== osd_op_reply(120 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [call out=124b] v0'0 uv15 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.285+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:121 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a22c00 con 0x5619658af000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.287+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(121 foo.buckets [call] v31'36 uv36 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.287+0000 7f389341f880 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.287+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:122 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23000 con 0x5619658af000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(122 myfoo [call,call,delete] v31'37 uv37 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f389341f880 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f389341f880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:123 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94667077130880 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23400 con 0x5619658afc00 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== watch-notify(notify (1) cookie 94667077125120 notify 133143986197 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x561965935a00 con 0x5619658afc00 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f388998b640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986197 cookie 94667077125120 notifier 4709 bl.length()=149 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 33 ==== osd_op_reply(123 notify.6 [notify cookie 94667077130880 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f388998b640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:124 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d000 con 0x5619658afc00 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 34 ==== osd_op_reply(124 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 35 ==== watch-notify(notify_complete (2) cookie 94667077130880 notify 133143986197 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561965935ba0 con 0x5619658afc00 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.288+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:125 8.4 8:26caafa4:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23800 con 0x5619658af000 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.289+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(125 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [call,call,delete] v31'3 uv3 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.289+0000 7f389341f880 10 removing default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 from cache 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.289+0000 7f389341f880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.5, ofs0, ns] 2024-04-27T05:21:57.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.289+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:126 7.15 7:a93a5511:::notify.2:head [notify cookie 94667077130880 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23c00 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 30 ==== watch-notify(notify (1) cookie 94667077120512 notify 133143986193 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x561966438ea0 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f388998b640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986193 cookie 94667077120512 notifier 4709 bl.length()=206 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 31 ==== osd_op_reply(126 notify.2 [notify cookie 94667077130880 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f388998b640 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:127 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0d400 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 32 ==== watch-notify(notify_complete (2) cookie 94667077130880 notify 133143986193 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561966439040 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:128 4.9 4:928c95c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0c800 con 0x561966329000 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:129 4.8 4:11b174e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0cc00 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 33 ==== osd_op_reply(127 notify.2 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:130 4.2f 4:f5dc11c0:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a0dc00 con 0x561966329000 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:131 4.33 4:ce65ea85:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16c00 con 0x5619658afc00 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:132 4.13 4:ca212683:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17000 con 0x5619658af000 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:133 4.37 4:ecf90668:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561966329400 con 0x5619658afc00 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:134 4.26 4:66f6476b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6400 con 0x5619658afc00 2024-04-27T05:21:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:135 4.3b 4:dcdfdbe5:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597dc00 con 0x5619658af800 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:136 4.12 4:4933ce3f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a22c00 con 0x561966329000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:137 4.27 4:e5072d9d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23000 con 0x561966329000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.290+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:138 4.3 4:c0dc6532:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23800 con 0x561966329000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.291+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 46 ==== osd_op_reply(128 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.0 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 47 ==== osd_op_reply(130 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.2 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 36 ==== osd_op_reply(133 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.5 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 37 ==== osd_op_reply(134 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.6 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 34 ==== osd_op_reply(135 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.7 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 38 ==== osd_op_reply(131 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.3 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 35 ==== osd_op_reply(129 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.1 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(132 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.4 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 48 ==== osd_op_reply(137 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.9 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.292+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 49 ==== osd_op_reply(136 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.8 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 50 ==== osd_op_reply(138 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.5.10 [delete] v31'16 uv16 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 : miss 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:139 6.13 6:cecfef72:::pubsub..bucket.myfoo%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a22000 con 0x5619658afc00 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 39 ==== osd_op_reply(139 pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 info.flags=0x0 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 10 adding default.rgw.log++pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 to cache LRU end 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.293+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:140 6.13 6:cecfef72:::pubsub..bucket.myfoo%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a23400 con 0x5619658afc00 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.294+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 40 ==== osd_op_reply(140 pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.5 [delete] v31'30 uv25 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.294+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:141 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d6c00 con 0x5619658af000 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.295+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(141 foo.buckets [call] v31'38 uv38 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.296+0000 7f3876965640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.296+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94667077127424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5619658d7000 con 0x561966329000 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.297+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 51 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94667077127424] v31'192 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.297+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:143 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94667077118208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597c800 con 0x5619658af000 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.298+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(143 notify.0 [watch unwatch cookie 94667077118208] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.298+0000 7f389341f880 20 remove_watcher() i=0 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.298+0000 7f389341f880 2 removed watcher, disabling cache 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.298+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:144 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94667077119360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d000 con 0x561966329000 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.299+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 52 ==== osd_op_reply(144 notify.1 [watch unwatch cookie 94667077119360] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.299+0000 7f389341f880 20 remove_watcher() i=1 2024-04-27T05:21:57.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.299+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:145 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94667077120512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56196597d400 con 0x5619658af800 2024-04-27T05:21:57.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.300+0000 7f388f997640 1 -- 172.21.15.138:0/454303614 <== osd.3 v2:172.21.15.138:6808/29866 36 ==== osd_op_reply(145 notify.2 [watch unwatch cookie 94667077120512] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bcfc0 con 0x5619658af800 2024-04-27T05:21:57.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.300+0000 7f389341f880 20 remove_watcher() i=2 2024-04-27T05:21:57.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.300+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:146 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94667077121664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17800 con 0x5619658af000 2024-04-27T05:21:57.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.301+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 <== osd.2 v2:172.21.15.138:6809/29865 24 ==== osd_op_reply(146 notify.3 [watch unwatch cookie 94667077121664] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bd200 con 0x5619658af000 2024-04-27T05:21:57.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.301+0000 7f389341f880 20 remove_watcher() i=3 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.301+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:147 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94667077122816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16800 con 0x561966329000 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.302+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.1 v2:172.21.15.138:6800/29864 53 ==== osd_op_reply(147 notify.4 [watch unwatch cookie 94667077122816] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x561966329000 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.302+0000 7f389341f880 20 remove_watcher() i=4 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.302+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:148 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94667077123968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17400 con 0x5619658afc00 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.303+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 41 ==== osd_op_reply(148 notify.5 [watch unwatch cookie 94667077123968] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.303+0000 7f389341f880 20 remove_watcher() i=5 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.303+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:149 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94667077125120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a16000 con 0x5619658afc00 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.304+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 42 ==== osd_op_reply(149 notify.6 [watch unwatch cookie 94667077125120] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.304+0000 7f389341f880 20 remove_watcher() i=6 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.304+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:150 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94667077126272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561965a17c00 con 0x5619658afc00 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f3890198640 1 -- 172.21.15.138:0/454303614 <== osd.0 v2:172.21.15.114:6800/27747 43 ==== osd_op_reply(150 notify.7 [watch unwatch cookie 94667077126272] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619658bc900 con 0x5619658afc00 2024-04-27T05:21:57.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 20 remove_watcher() i=7 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619658afc00 msgr2=0x561965819180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5619658afc00 0x561965819180 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966329000 msgr2=0x561965817b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966329000 0x561965817b80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619658af000 msgr2=0x561965818c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5619658af000 0x561965818c00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619658af800 msgr2=0x561965818100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5619658af800 0x561965818100 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619658ae000 msgr2=0x561965817600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5619658ae000 0x561965817600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x561966431260 tx=0x56196648ee40 comp rx=0 tx=0).stop 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 msgr2=0x561965816b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 --2- 172.21.15.138:0/454303614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561966328c00 0x561965816b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x5619663e5980 tx=0x561965647140 comp rx=0 tx=0).stop 2024-04-27T05:21:57.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 reap_dead start 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f388f196640 1 -- 172.21.15.138:0/454303614 reap_dead start 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 shutdown_connections 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 >> 172.21.15.138:0/454303614 conn(0x561965535400 msgr2=0x56196563f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 shutdown_connections 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.305+0000 7f389341f880 1 -- 172.21.15.138:0/454303614 wait complete. 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56196597c400 msgr2=0x561965978580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56196597c400 0x561965978580 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966490800 msgr2=0x561965978000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561966490800 0x561965978000 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56196597d800 msgr2=0x561965979080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56196597d800 0x561965979080 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56196597cc00 msgr2=0x561965978b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56196597cc00 0x561965978b00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966490c00 msgr2=0x561965819700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966490c00 0x561965819700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x56196588d200 tx=0x56196648f320 comp rx=0 tx=0).stop 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 msgr2=0x561965818680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/492702467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5619658aec00 0x561965818680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x56196588d890 tx=0x56196648f560 comp rx=0 tx=0).stop 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f388f997640 1 -- 172.21.15.138:0/492702467 reap_dead start 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 shutdown_connections 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 >> 172.21.15.138:0/492702467 conn(0x5619658d7800 msgr2=0x56196563f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 shutdown_connections 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/492702467 wait complete. 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966329c00 msgr2=0x561965817080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966329c00 0x561965817080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5619664302a0 tx=0x561965647c20 comp rx=0 tx=0).stop 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 msgr2=0x561965816580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x5619663e4de0 tx=0x561965646c60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 shutdown_connections 2024-04-27T05:21:57.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561966329c00 0x561965817080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 --2- 172.21.15.138:0/111934428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561965535000 0x561965816580 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 >> 172.21.15.138:0/111934428 conn(0x561965534c00 msgr2=0x56196563d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 shutdown_connections 2024-04-27T05:21:57.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.306+0000 7f389341f880 1 -- 172.21.15.138:0/111934428 wait complete. 2024-04-27T05:21:57.338 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T05:21:57.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T05:21:57.339 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-04-27T05:21:57.417 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:57.418 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:57.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f0621738880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55765cadf400 0x55765cdc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f0621738880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55765cdb6600 con 0x55765cadf000 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f0621738880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55765cb1e280 con 0x55765cadf400 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061d913640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55765cadf400 0x55765cdc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061e114640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55765cadf000 0x55765cbe6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:45078/0 (socket says 172.21.15.138:45078) 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061e114640 1 -- 172.21.15.138:0/1746543971 learned_addr learned my addr 172.21.15.138:0/1746543971 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061d112640 1 -- 172.21.15.138:0/1746543971 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732459528 0 0) 0x55765cae29c0 con 0x55765cadf000 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061d112640 1 -- 172.21.15.138:0/1746543971 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55765cdb7980 con 0x55765cadf000 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061d913640 1 -- 172.21.15.138:0/1746543971 >> v1:172.21.15.114:6789/0 conn(0x55765cadf000 legacy=0x55765cbe6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.417+0000 7f061d913640 1 -- 172.21.15.138:0/1746543971 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cae3380 con 0x55765cadf400 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.419+0000 7f061d913640 1 --2- 172.21.15.138:0/1746543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf400 0x55765cdc0580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55765cbeac00 tx=0x55765cbf0e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d38941d4626fd30 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.419+0000 7f061d112640 1 -- 172.21.15.138:0/1746543971 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf7ce0 con 0x55765cadf400 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.419+0000 7f061d112640 1 -- 172.21.15.138:0/1746543971 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765cdb6600 con 0x55765cadf400 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.419+0000 7f061d112640 1 -- 172.21.15.138:0/1746543971 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf7e40 con 0x55765cadf400 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 -- 172.21.15.138:0/1746543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf400 msgr2=0x55765cdc0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 --2- 172.21.15.138:0/1746543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf400 0x55765cdc0580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55765cbeac00 tx=0x55765cbf0e40 comp rx=0 tx=0).stop 2024-04-27T05:21:57.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 -- 172.21.15.138:0/1746543971 shutdown_connections 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 --2- 172.21.15.138:0/1746543971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf400 0x55765cdc0580 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 -- 172.21.15.138:0/1746543971 >> 172.21.15.138:0/1746543971 conn(0x55765cadec00 msgr2=0x55765cbe6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 -- 172.21.15.138:0/1746543971 shutdown_connections 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 -- 172.21.15.138:0/1746543971 wait complete. 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.420+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f0621738880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f0621738880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765cb1e780 con 0x55765cadf000 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061e114640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061e114640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45036/0 (socket says 172.21.15.138:45036) 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061e114640 1 -- 172.21.15.138:0/2961797747 learned_addr learned my addr 172.21.15.138:0/2961797747 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061e114640 1 -- 172.21.15.138:0/2961797747 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cae2ea0 con 0x55765cadf000 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061e114640 1 --2- 172.21.15.138:0/2961797747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55765d996c90 tx=0x55765cbf0ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbf1f00c61e9495 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061c911640 1 -- 172.21.15.138:0/2961797747 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf74a0 con 0x55765cadf000 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061c911640 1 -- 172.21.15.138:0/2961797747 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765cdb7e00 con 0x55765cadf000 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.421+0000 7f061c911640 1 -- 172.21.15.138:0/2961797747 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf7ce0 con 0x55765cadf000 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- 172.21.15.138:0/2961797747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 msgr2=0x55765cdc0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 --2- 172.21.15.138:0/2961797747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x55765d996c90 tx=0x55765cbf0ea0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- 172.21.15.138:0/2961797747 shutdown_connections 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 --2- 172.21.15.138:0/2961797747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- 172.21.15.138:0/2961797747 >> 172.21.15.138:0/2961797747 conn(0x55765cadec00 msgr2=0x55765cbe7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- 172.21.15.138:0/2961797747 shutdown_connections 2024-04-27T05:21:57.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- 172.21.15.138:0/2961797747 wait complete. 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f0621738880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765cb1e8c0 con 0x55765cadf000 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f061e114640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f061e114640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45040/0 (socket says 172.21.15.138:45040) 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.422+0000 7f061e114640 1 -- 172.21.15.138:0/3407626981 learned_addr learned my addr 172.21.15.138:0/3407626981 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f061e114640 1 -- 172.21.15.138:0/3407626981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cae36c0 con 0x55765cadf000 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f061e114640 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55765d997350 tx=0x55765cbf0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f061b10e640 1 -- 172.21.15.138:0/3407626981 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf7e40 con 0x55765cadf000 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f061b10e640 1 -- 172.21.15.138:0/3407626981 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765d9a0000 con 0x55765cadf000 2024-04-27T05:21:57.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55765cae31e0 con 0x55765cadf000 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f061b10e640 1 -- 172.21.15.138:0/3407626981 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf74a0 con 0x55765cadf000 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.423+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55765cb50ea0 con 0x55765cadf000 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061b10e640 1 -- 172.21.15.138:0/3407626981 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55765cb09600 con 0x55765cadf000 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f0621738880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f0621738880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765cb02280 con 0x55765d8d7800 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061b10e640 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765da30000 0x55765cdc1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061b10e640 1 -- 172.21.15.138:0/3407626981 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55765caf1a40 con 0x55765cadf000 2024-04-27T05:21:57.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45048/0 (socket says 172.21.15.138:45048) 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e915640 1 -- 172.21.15.138:0/1011181507 learned_addr learned my addr 172.21.15.138:0/1011181507 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e114640 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765da30000 0x55765cdc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e915640 1 -- 172.21.15.138:0/1011181507 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cb50d00 con 0x55765d8d7800 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e114640 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765da30000 0x55765cdc1080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55765d9e4420 tx=0x55765cbf14a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f061e915640 1 --2- 172.21.15.138:0/1011181507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55765d997e90 tx=0x55765cbf1c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cfbaf53abb27af server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.424+0000 7f0618909640 1 -- 172.21.15.138:0/1011181507 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf7e40 con 0x55765d8d7800 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0618909640 1 -- 172.21.15.138:0/1011181507 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765d9a0480 con 0x55765d8d7800 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0618909640 1 -- 172.21.15.138:0/1011181507 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf74a0 con 0x55765d8d7800 2024-04-27T05:21:57.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- 172.21.15.138:0/1011181507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 msgr2=0x55765cdc0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 --2- 172.21.15.138:0/1011181507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55765d997e90 tx=0x55765cbf1c80 comp rx=0 tx=0).stop 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- 172.21.15.138:0/1011181507 shutdown_connections 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 --2- 172.21.15.138:0/1011181507 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- 172.21.15.138:0/1011181507 >> 172.21.15.138:0/1011181507 conn(0x55765cadf400 msgr2=0x55765cbe9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- 172.21.15.138:0/1011181507 shutdown_connections 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- 172.21.15.138:0/1011181507 wait complete. 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f0621738880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765cb03180 con 0x55765d8d7800 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f061e915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f061e915640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45064/0 (socket says 172.21.15.138:45064) 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.425+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 learned_addr learned my addr 172.21.15.138:0/3995956642 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cb50b60 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f061e915640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55765d997980 tx=0x55765cbf1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f0617106640 1 -- 172.21.15.138:0/3995956642 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf7e40 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f0617106640 1 -- 172.21.15.138:0/3995956642 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765d9a0600 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55765cb509c0 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f0617106640 1 -- 172.21.15.138:0/3995956642 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765caf74a0 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.426+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55765cb50820 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f0617106640 1 -- 172.21.15.138:0/3995956642 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55765cb098c0 con 0x55765d8d7800 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f0617106640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765ce70000 0x55765cdc1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f0617106640 1 -- 172.21.15.138:0/3995956642 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55765caf1c00 con 0x55765d8d7800 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f061d913640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765ce70000 0x55765cdc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55765d8d7c00 0x55765cdc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70400 con 0x55765d8d7c00 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f061e915640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55765d8d7c00 0x55765cdc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f061d913640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765ce70000 0x55765cdc1600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55765cbeac00 tx=0x55765da60d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.427+0000 7f061e915640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55765d8d7c00 0x55765cdc1b80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 realm 2024-04-27T05:21:57.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70800 con 0x55765d8d7c00 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70c00 con 0x55765d8d7c00 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71000 con 0x55765d8d7c00 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55765ce71400 0x55765cdc2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70400 con 0x55765ce71400 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.428+0000 7f061e114640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55765ce71400 0x55765cdc2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f061e114640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55765ce71400 0x55765cdc2100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f0621738880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f0621738880 20 searching for the correct realm 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70800 con 0x55765ce71400 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f061e114640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55765ce70800 0x55765cdc2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70400 con 0x55765ce70800 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.429+0000 7f061d913640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55765ce70800 0x55765cdc2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.430+0000 7f061d913640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55765ce70800 0x55765cdc2c00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.430+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.430+0000 7f061d913640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55765ce70400 0x55765cdc3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.430+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.430+0000 7f061e915640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55765ce70400 0x55765cdc3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.430+0000 7f061e915640 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55765ce70400 0x55765cdc3180 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.431+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70800 2024-04-27T05:21:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.432+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70800 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.433+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.434+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70800 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.435+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70800 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce71400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.436+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765d8d7c00 2024-04-27T05:21:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8d6800 con 0x55765ce70800 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70c00 con 0x55765d8d7c00 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.437+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71000 con 0x55765d8d7c00 2024-04-27T05:21:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71800 con 0x55765ce71400 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 zone default found 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 4 Realm: () 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 zonegroup default 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71c00 con 0x55765ce70400 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.438+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc000 con 0x55765ce71400 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc400 con 0x55765ce71400 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 20 started sync module instance, tier type = 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.439+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc800 con 0x55765ce70800 2024-04-27T05:21:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.440+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.440+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93966852393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8ccc00 con 0x55765ce70800 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.440+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cd000 con 0x55765d8d7c00 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.441+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93966852393216] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.441+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.441+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93966852394368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cd400 con 0x55765d8d7c00 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.441+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cd800 con 0x55765ce71400 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.442+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.442+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93966852395520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc000 con 0x55765ce71400 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.442+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc400 con 0x55765ce70800 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.442+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93966852394368] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.443+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93966852395520] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.443+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.443+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93966852396672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cd800 con 0x55765ce70800 2024-04-27T05:21:57.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.443+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71800 con 0x55765d8d7c00 2024-04-27T05:21:57.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.444+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93966852396672] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.444+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.444+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93966852397824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce70c00 con 0x55765d8d7c00 2024-04-27T05:21:57.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.444+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71000 con 0x55765ce70400 2024-04-27T05:21:57.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.445+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93966852397824] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.446+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.446+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93966852398976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765ce71c00 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.446+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765da31800 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.447+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93966852398976] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.447+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.447+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93966852400128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc800 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.447+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8ccc00 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93966852400128] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93966852401280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cc400 con 0x55765ce70400 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=0 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=1 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=2 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=3 2024-04-27T05:21:57.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=4 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=5 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.448+0000 7f0621738880 20 add_watcher() i=6 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.449+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93966852401280] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.449+0000 7f0621738880 20 add_watcher() i=7 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.449+0000 7f0621738880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.449+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cd800 con 0x55765d8d7c00 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.450+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.450+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93966852402432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765d8cd000 con 0x55765d8d7c00 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93966852402432] v31'193 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f06050e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f0621738880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f0621738880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:57.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f0621738880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.451+0000 7f0621738880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765cb02780 con 0x55765d8cdc00 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f061d913640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f061d913640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45070/0 (socket says 172.21.15.138:45070) 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f061d913640 1 -- 172.21.15.138:0/1479567020 learned_addr learned my addr 172.21.15.138:0/1479567020 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f061d913640 1 -- 172.21.15.138:0/1479567020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cb50680 con 0x55765d8cdc00 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f061d913640 1 --2- 172.21.15.138:0/1479567020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55765ce4ec30 tx=0x55765da60de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff8a13b9863d5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f06048e1640 1 -- 172.21.15.138:0/1479567020 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765ce8c580 con 0x55765d8cdc00 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f06048e1640 1 -- 172.21.15.138:0/1479567020 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765d9a0000 con 0x55765d8cdc00 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.452+0000 7f06048e1640 1 -- 172.21.15.138:0/1479567020 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765ce8c6e0 con 0x55765d8cdc00 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- 172.21.15.138:0/1479567020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 msgr2=0x55765cdc2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 --2- 172.21.15.138:0/1479567020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55765ce4ec30 tx=0x55765da60de0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- 172.21.15.138:0/1479567020 shutdown_connections 2024-04-27T05:21:57.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 --2- 172.21.15.138:0/1479567020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- 172.21.15.138:0/1479567020 >> 172.21.15.138:0/1479567020 conn(0x55765d8cd400 msgr2=0x55765cbe9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- 172.21.15.138:0/1479567020 shutdown_connections 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- 172.21.15.138:0/1479567020 wait complete. 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 Processor -- start 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- start start 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.453+0000 7f0621738880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765cb02500 con 0x55765d8cdc00 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f061d913640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f061d913640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:45084/0 (socket says 172.21.15.138:45084) 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f061d913640 1 -- 172.21.15.138:0/2743991613 learned_addr learned my addr 172.21.15.138:0/2743991613 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f061d913640 1 -- 172.21.15.138:0/2743991613 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765cae2ea0 con 0x55765d8cdc00 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f061d913640 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55765ce4f7a0 tx=0x55765da60c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f06030de640 1 -- 172.21.15.138:0/2743991613 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765ce8c840 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f06030de640 1 -- 172.21.15.138:0/2743991613 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55765d9a0900 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55765cae36c0 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f06030de640 1 -- 172.21.15.138:0/2743991613 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55765ce8c580 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.454+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55765cae31e0 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f06030de640 1 -- 172.21.15.138:0/2743991613 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55765cb08b00 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f06030de640 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765d8cc800 0x55765cdc3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f06030de640 1 -- 172.21.15.138:0/2743991613 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55765caf1dc0 con 0x55765d8cdc00 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f061e915640 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765d8cc800 0x55765cdc3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f0621738880 5 note: GC not initialized 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f06008d9640 20 reqs_thread_entry: start 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f0621738880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f061e915640 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765d8cc800 0x55765cdc3700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55765da58a20 tx=0x55765da610e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f0621738880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:57.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.455+0000 7f05fe8d5640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:27 2024 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f05fe0b3640 20 reqs_thread_entry: start 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf56000 con 0x55765d8d7c00 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.456+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~369 in=369b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf56800 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~369] v31'14 uv14 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0621738880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0621738880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0621738880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93966852404736 in=557b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf56c00 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 93966852397824 notify 133143986194 ret 0) v3 ==== 587+0+0 (crc 0 0 0) 0x55765ce35d40 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0618909640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986194 cookie 93966852397824 notifier 4717 bl.length()=545 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0618909640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0618909640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 93966852404736 out=8b] v0'0 uv180 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.457+0000 7f0618909640 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf6a000 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.458+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.458+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 93966852404736 notify 133143986194 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55765cedd860 con 0x55765d8d7c00 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:57.482 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [ 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:57.483 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "user", 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "read" 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:57.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:57.485 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:57.486 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:57.486 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:57.486 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.458+0000 7f06050e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.458+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93966852402432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf57000 con 0x55765d8d7c00 2024-04-27T05:21:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.459+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93966852402432] v31'194 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.459+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93966852393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf57400 con 0x55765ce70800 2024-04-27T05:21:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.460+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93966852393216] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.460+0000 7f0621738880 20 remove_watcher() i=0 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.460+0000 7f0621738880 2 removed watcher, disabling cache 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.460+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93966852394368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf57800 con 0x55765d8d7c00 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.461+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93966852394368] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.461+0000 7f0621738880 20 remove_watcher() i=1 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.461+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93966852395520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf57c00 con 0x55765ce71400 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.462+0000 7f061e114640 1 -- 172.21.15.138:0/3995956642 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93966852395520] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82fc0 con 0x55765ce71400 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.462+0000 7f0621738880 20 remove_watcher() i=2 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.462+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93966852396672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf6a000 con 0x55765ce70800 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.463+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93966852396672] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce83200 con 0x55765ce70800 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.463+0000 7f0621738880 20 remove_watcher() i=3 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.463+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93966852397824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf6a800 con 0x55765d8d7c00 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.464+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93966852397824] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765d8d7c00 2024-04-27T05:21:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.464+0000 7f0621738880 20 remove_watcher() i=4 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.464+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93966852398976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf6ac00 con 0x55765ce70400 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.465+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93966852398976] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.465+0000 7f0621738880 20 remove_watcher() i=5 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.465+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93966852400128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf6b000 con 0x55765ce70400 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.466+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93966852400128] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.466+0000 7f0621738880 20 remove_watcher() i=6 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.466+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93966852401280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55765cf6b400 con 0x55765ce70400 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f061e915640 1 -- 172.21.15.138:0/3995956642 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93966852401280] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55765ce82900 con 0x55765ce70400 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 20 remove_watcher() i=7 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55765ce70400 msgr2=0x55765cdc3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55765ce70400 0x55765cdc3180 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55765d8d7c00 msgr2=0x55765cdc1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55765d8d7c00 0x55765cdc1b80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55765ce70800 msgr2=0x55765cdc2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55765ce70800 0x55765cdc2c00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55765ce71400 msgr2=0x55765cdc2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55765ce71400 0x55765cdc2100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765ce70000 msgr2=0x55765cdc1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765ce70000 0x55765cdc1600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55765cbeac00 tx=0x55765da60d80 comp rx=0 tx=0).stop 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 msgr2=0x55765cdc0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 --2- 172.21.15.138:0/3995956642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8d7800 0x55765cdc0b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55765d997980 tx=0x55765cbf1a40 comp rx=0 tx=0).stop 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 reap_dead start 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f061d913640 1 -- 172.21.15.138:0/3995956642 reap_dead start 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 shutdown_connections 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 >> 172.21.15.138:0/3995956642 conn(0x55765cadf400 msgr2=0x55765cbe9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 shutdown_connections 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.467+0000 7f0621738880 1 -- 172.21.15.138:0/3995956642 wait complete. 2024-04-27T05:21:57.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765d8cc800 msgr2=0x55765cdc3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765d8cc800 0x55765cdc3700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55765da58a20 tx=0x55765da610e0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 msgr2=0x55765cdc2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55765ce4f7a0 tx=0x55765da60c60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 shutdown_connections 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765d8cc800 0x55765cdc3700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/2743991613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765d8cdc00 0x55765cdc2680 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 >> 172.21.15.138:0/2743991613 conn(0x55765d8cd400 msgr2=0x55765cbe9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 shutdown_connections 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/2743991613 wait complete. 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765da30000 msgr2=0x55765cdc1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765da30000 0x55765cdc1080 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55765d9e4420 tx=0x55765cbf14a0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 msgr2=0x55765cdc0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55765d997350 tx=0x55765cbf0d20 comp rx=0 tx=0).stop 2024-04-27T05:21:57.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 shutdown_connections 2024-04-27T05:21:57.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55765da30000 0x55765cdc1080 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 --2- 172.21.15.138:0/3407626981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55765cadf000 0x55765cdc0580 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 >> 172.21.15.138:0/3407626981 conn(0x55765cadec00 msgr2=0x55765cbe7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 shutdown_connections 2024-04-27T05:21:57.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.468+0000 7f0621738880 1 -- 172.21.15.138:0/3407626981 wait complete. 2024-04-27T05:21:57.498 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:57.498 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T05:21:57.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T05:21:57.499 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-04-27T05:21:57.544 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:57.545 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:57.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3e1f30880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3e1f30880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5610d8d6e600 con 0x5610d8a97400 2024-04-27T05:21:57.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3e1f30880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5610d8ad6280 con 0x5610d8a97000 2024-04-27T05:21:57.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3de854640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3de053640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5610d8a97400 0x5610d8b9e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42784/0 (socket says 172.21.15.138:42784) 2024-04-27T05:21:57.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3de053640 1 -- 172.21.15.138:0/4026929168 learned_addr learned my addr 172.21.15.138:0/4026929168 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:57.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.544+0000 7ff3de854640 1 -- 172.21.15.138:0/4026929168 >> v1:172.21.15.114:6789/0 conn(0x5610d8a97400 legacy=0x5610d8b9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.545+0000 7ff3de854640 1 -- 172.21.15.138:0/4026929168 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8a9b380 con 0x5610d8a97000 2024-04-27T05:21:57.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.545+0000 7ff3dd852640 1 -- 172.21.15.138:0/4026929168 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176462499 0 0) 0x5610d8a9a9c0 con 0x5610d8a97400 2024-04-27T05:21:57.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.546+0000 7ff3de854640 1 --2- 172.21.15.138:0/4026929168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5610d8ba2d50 tx=0x5610d8ba8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b76b1496d2c82e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.546+0000 7ff3dd852640 1 -- 172.21.15.138:0/4026929168 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aafce0 con 0x5610d8a97000 2024-04-27T05:21:57.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.546+0000 7ff3dd852640 1 -- 172.21.15.138:0/4026929168 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d8d6fb00 con 0x5610d8a97000 2024-04-27T05:21:57.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.546+0000 7ff3dd852640 1 -- 172.21.15.138:0/4026929168 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aafe40 con 0x5610d8a97000 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 -- 172.21.15.138:0/4026929168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 msgr2=0x5610d8d78580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/4026929168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5610d8ba2d50 tx=0x5610d8ba8de0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 -- 172.21.15.138:0/4026929168 shutdown_connections 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/4026929168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 -- 172.21.15.138:0/4026929168 >> 172.21.15.138:0/4026929168 conn(0x5610d8a96c00 msgr2=0x5610d8b9e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 -- 172.21.15.138:0/4026929168 shutdown_connections 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 -- 172.21.15.138:0/4026929168 wait complete. 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.547+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3e1f30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3e1f30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8ad6780 con 0x5610d8a97000 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3de854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3de854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:60970/0 (socket says 172.21.15.138:60970) 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3de854640 1 -- 172.21.15.138:0/330669177 learned_addr learned my addr 172.21.15.138:0/330669177 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3de854640 1 -- 172.21.15.138:0/330669177 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8a9aea0 con 0x5610d8a97000 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3de854640 1 --2- 172.21.15.138:0/330669177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5610d989d590 tx=0x5610d8ba8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c57eb778f258db server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3dd051640 1 -- 172.21.15.138:0/330669177 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aaf4a0 con 0x5610d8a97000 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3dd051640 1 -- 172.21.15.138:0/330669177 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d8d6fe00 con 0x5610d8a97000 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.548+0000 7ff3dd051640 1 -- 172.21.15.138:0/330669177 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aafce0 con 0x5610d8a97000 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- 172.21.15.138:0/330669177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 msgr2=0x5610d8d78580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/330669177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5610d989d590 tx=0x5610d8ba8cc0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- 172.21.15.138:0/330669177 shutdown_connections 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/330669177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- 172.21.15.138:0/330669177 >> 172.21.15.138:0/330669177 conn(0x5610d8a96c00 msgr2=0x5610d8b9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- 172.21.15.138:0/330669177 shutdown_connections 2024-04-27T05:21:57.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- 172.21.15.138:0/330669177 wait complete. 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.549+0000 7ff3e1f30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8ad68c0 con 0x5610d8a97000 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3de854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3de854640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:60976/0 (socket says 172.21.15.138:60976) 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3de854640 1 -- 172.21.15.138:0/1059610390 learned_addr learned my addr 172.21.15.138:0/1059610390 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3de854640 1 -- 172.21.15.138:0/1059610390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8a9b6c0 con 0x5610d8a97000 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3de854640 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5610d9952e10 tx=0x5610d8ba8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3db84e640 1 -- 172.21.15.138:0/1059610390 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aafe40 con 0x5610d8a97000 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3db84e640 1 -- 172.21.15.138:0/1059610390 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d9876000 con 0x5610d8a97000 2024-04-27T05:21:57.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3db84e640 1 -- 172.21.15.138:0/1059610390 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aaf4a0 con 0x5610d8a97000 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610d8a9b1e0 con 0x5610d8a97000 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.550+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610d8b08ea0 con 0x5610d8a97000 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3db84e640 1 -- 172.21.15.138:0/1059610390 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5610d8ac0b00 con 0x5610d8a97000 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3e1f30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3e1f30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8aba280 con 0x5610d988ec00 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3db84e640 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d988fc00 0x5610d8d79080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3db84e640 1 -- 172.21.15.138:0/1059610390 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5610d8aa9a40 con 0x5610d8a97000 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3df055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3df055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:60978/0 (socket says 172.21.15.138:60978) 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3df055640 1 -- 172.21.15.138:0/1344204883 learned_addr learned my addr 172.21.15.138:0/1344204883 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3de854640 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d988fc00 0x5610d8d79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3df055640 1 -- 172.21.15.138:0/1344204883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8b08d00 con 0x5610d988ec00 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.551+0000 7ff3de854640 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d988fc00 0x5610d8d79080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5610d99982d0 tx=0x5610d8ba9c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3df055640 1 --2- 172.21.15.138:0/1344204883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5610d9953d40 tx=0x5610d8ba9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1e4c5c01d781dd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3d9049640 1 -- 172.21.15.138:0/1344204883 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aafe40 con 0x5610d988ec00 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3d9049640 1 -- 172.21.15.138:0/1344204883 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d9876600 con 0x5610d988ec00 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3d9049640 1 -- 172.21.15.138:0/1344204883 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aaf4a0 con 0x5610d988ec00 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1344204883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 msgr2=0x5610d8d78b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1344204883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5610d9953d40 tx=0x5610d8ba9f80 comp rx=0 tx=0).stop 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1344204883 shutdown_connections 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1344204883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1344204883 >> 172.21.15.138:0/1344204883 conn(0x5610d8a97400 msgr2=0x5610d8ba1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1344204883 shutdown_connections 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1344204883 wait complete. 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.552+0000 7ff3e1f30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8abb180 con 0x5610d988ec00 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3df055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3df055640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:60980/0 (socket says 172.21.15.138:60980) 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 learned_addr learned my addr 172.21.15.138:0/818775627 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8b08b60 con 0x5610d988ec00 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3df055640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5610d9953950 tx=0x5610d8ba9140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3d7846640 1 -- 172.21.15.138:0/818775627 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aafe40 con 0x5610d988ec00 2024-04-27T05:21:57.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3d7846640 1 -- 172.21.15.138:0/818775627 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d9876780 con 0x5610d988ec00 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610d8b089c0 con 0x5610d988ec00 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3d7846640 1 -- 172.21.15.138:0/818775627 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8aaf4a0 con 0x5610d988ec00 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.553+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610d8b08820 con 0x5610d988ec00 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3d7846640 1 -- 172.21.15.138:0/818775627 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5610d8ac1600 con 0x5610d988ec00 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3d7846640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d8e14000 0x5610d8d79600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3d7846640 1 -- 172.21.15.138:0/818775627 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5610d8aa9c00 con 0x5610d988ec00 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3de053640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d8e14000 0x5610d8d79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5610d988f000 0x5610d8d79b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d988f400 con 0x5610d988f000 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3df055640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5610d988f000 0x5610d8d79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.554+0000 7ff3de053640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d8e14000 0x5610d8d79600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5610d8ba3410 tx=0x5610d99f2f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3df055640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5610d988f000 0x5610d8d79b80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 realm 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d988f800 con 0x5610d988f000 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15000 con 0x5610d988f000 2024-04-27T05:21:57.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15400 con 0x5610d988f000 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.555+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5610d8e15800 0x5610d8d7a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15c00 con 0x5610d8e15800 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3de854640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5610d8e15800 0x5610d8d7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3de854640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5610d8e15800 0x5610d8d7a100 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3e1f30880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3e1f30880 20 searching for the correct realm 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15000 con 0x5610d8e15800 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3de854640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5610d8e15000 0x5610d8d7ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.556+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15c00 con 0x5610d8e15000 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.557+0000 7ff3de053640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5610d8e15000 0x5610d8d7ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.557+0000 7ff3de053640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5610d8e15000 0x5610d8d7ac00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.557+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.557+0000 7ff3de053640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5610d8e15c00 0x5610d8d7b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.557+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.557+0000 7ff3df055640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5610d8e15c00 0x5610d8d7b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5610d8e15c00 0x5610d8d7b180 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.558+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15000 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.559+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15000 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.560+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.561+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15000 2024-04-27T05:21:57.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.562+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.563+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.563+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.563+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.563+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15000 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15800 2024-04-27T05:21:57.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15c00 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.564+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d988f000 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e14c00 con 0x5610d8e15000 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:57.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15400 con 0x5610d988f000 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d988f400 con 0x5610d988f000 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.565+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d988f800 con 0x5610d8e15800 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 zone default found 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 4 Realm: () 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 zonegroup default 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a000 con 0x5610d8e15c00 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a400 con 0x5610d8e15800 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a800 con 0x5610d8e15800 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 started sync module instance, tier type = 2024-04-27T05:21:57.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.566+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3ac00 con 0x5610d8e15000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.567+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.567+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94630357436672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3b000 con 0x5610d8e15000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.567+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3b400 con 0x5610d988f000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.568+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94630357436672] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.568+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.569+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94630357437824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3b800 con 0x5610d988f000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.569+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3bc00 con 0x5610d8e15800 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.569+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94630357437824] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.570+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.570+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94630357438976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a400 con 0x5610d8e15800 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.570+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a800 con 0x5610d8e15000 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.570+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94630357438976] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.571+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.571+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94630357440128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3bc00 con 0x5610d8e15000 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.571+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a000 con 0x5610d988f000 2024-04-27T05:21:57.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.572+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.572+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94630357441280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d988f800 con 0x5610d988f000 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.572+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d988f400 con 0x5610d8e15c00 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.573+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94630357440128] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.573+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.573+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94630357442432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e15400 con 0x5610d8e15c00 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.573+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d99f5800 con 0x5610d8e15c00 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.573+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94630357441280] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.574+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94630357442432] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.574+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.574+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94630357443584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d99f5400 con 0x5610d8e15c00 2024-04-27T05:21:57.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.574+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3ac00 con 0x5610d8e15c00 2024-04-27T05:21:57.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94630357444736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3b000 con 0x5610d8e15c00 2024-04-27T05:21:57.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94630357443584] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=0 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=1 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=2 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=3 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=4 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=5 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.576+0000 7ff3e1f30880 20 add_watcher() i=6 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.577+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94630357444736] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.577+0000 7ff3e1f30880 20 add_watcher() i=7 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.577+0000 7ff3e1f30880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.577+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3a800 con 0x5610d988f000 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.577+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.578+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94630357445888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8e3b400 con 0x5610d988f000 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94630357445888] v31'195 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3c5822640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3e1f30880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3e1f30880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3e1f30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3e1f30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8aba780 con 0x5610d8e14c00 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3de053640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3de053640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:60984/0 (socket says 172.21.15.138:60984) 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.579+0000 7ff3de053640 1 -- 172.21.15.138:0/334015517 learned_addr learned my addr 172.21.15.138:0/334015517 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.580+0000 7ff3de053640 1 -- 172.21.15.138:0/334015517 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8b08680 con 0x5610d8e14c00 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.580+0000 7ff3de053640 1 --2- 172.21.15.138:0/334015517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5610d8df7620 tx=0x5610d99f2f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97f93e027900ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.580+0000 7ff3c5021640 1 -- 172.21.15.138:0/334015517 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8e3c580 con 0x5610d8e14c00 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.580+0000 7ff3c5021640 1 -- 172.21.15.138:0/334015517 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d9876000 con 0x5610d8e14c00 2024-04-27T05:21:57.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.580+0000 7ff3c5021640 1 -- 172.21.15.138:0/334015517 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8e3c6e0 con 0x5610d8e14c00 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- 172.21.15.138:0/334015517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 msgr2=0x5610d8d7a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/334015517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5610d8df7620 tx=0x5610d99f2f00 comp rx=0 tx=0).stop 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- 172.21.15.138:0/334015517 shutdown_connections 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/334015517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- 172.21.15.138:0/334015517 >> 172.21.15.138:0/334015517 conn(0x5610d8e3b800 msgr2=0x5610d8ba1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- 172.21.15.138:0/334015517 shutdown_connections 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- 172.21.15.138:0/334015517 wait complete. 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 Processor -- start 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- start start 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3e1f30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8aba500 con 0x5610d8e14c00 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3de053640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3de053640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:60990/0 (socket says 172.21.15.138:60990) 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.581+0000 7ff3de053640 1 -- 172.21.15.138:0/1160667029 learned_addr learned my addr 172.21.15.138:0/1160667029 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3de053640 1 -- 172.21.15.138:0/1160667029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d8b084e0 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3de053640 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5610d8df7830 tx=0x5610d99f3080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3c381e640 1 -- 172.21.15.138:0/1160667029 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8e3c840 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3c381e640 1 -- 172.21.15.138:0/1160667029 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5610d9876a80 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610d8a9b380 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3c381e640 1 -- 172.21.15.138:0/1160667029 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610d8e3c580 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.582+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610d8a9aea0 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3c381e640 1 -- 172.21.15.138:0/1160667029 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5610d8ac1b80 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3c381e640 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d99f5800 0x5610d8d7b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3c381e640 1 -- 172.21.15.138:0/1160667029 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5610d8aa9dc0 con 0x5610d8e14c00 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3e1f30880 5 note: GC not initialized 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3df055640 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d99f5800 0x5610d8d7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3c1019640 20 reqs_thread_entry: start 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3e1f30880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3df055640 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d99f5800 0x5610d8d7b700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5610d8df7170 tx=0x5610d99f3ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3e1f30880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3bf015640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:27 2024 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3be7f3640 20 reqs_thread_entry: start 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3e1f30880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3e1f30880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.583+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0c000 con 0x5610d988f000 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.584+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~369 out=369b,getxattrs out=75b] v0'0 uv14 ondisk = 0) v8 ==== 231+0+492 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.584+0000 7ff3e1f30880 20 rados_obj.operate() r=0 bl.length=369 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.584+0000 7ff3e1f30880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.584+0000 7ff3e1f30880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.584+0000 7ff3e1f30880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.584+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0c800 con 0x5610d988f000 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v31'15 uv15 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3e1f30880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3e1f30880 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3e1f30880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94630357448192 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0cc00 con 0x5610d988f000 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== watch-notify(notify (1) cookie 94630357441280 notify 133143986195 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5610d8ddbd40 con 0x5610d988f000 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3d9049640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986195 cookie 94630357441280 notifier 4725 bl.length()=533 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(64 notify.4 [notify cookie 94630357448192 out=8b] v0'0 uv183 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3d9049640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3d9049640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.585+0000 7ff3d9049640 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f20000 con 0x5610d988f000 2024-04-27T05:21:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.586+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.586+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== watch-notify(notify_complete (2) cookie 94630357448192 notify 133143986195 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5610d8e93a00 con 0x5610d988f000 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "foo", 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Foo", 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "foo@foo.com", 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:21:57.614 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:21:57.615 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:21:57.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:21:57.617 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:21:57.617 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:21:57.617 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:21:57.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.586+0000 7ff3c5822640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.586+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94630357445888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0d000 con 0x5610d988f000 2024-04-27T05:21:57.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.587+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94630357445888] v31'196 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.587+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94630357436672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0d400 con 0x5610d8e15000 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.588+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94630357436672] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.588+0000 7ff3e1f30880 20 remove_watcher() i=0 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.588+0000 7ff3e1f30880 2 removed watcher, disabling cache 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.588+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94630357437824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0d800 con 0x5610d988f000 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.589+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94630357437824] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.589+0000 7ff3e1f30880 20 remove_watcher() i=1 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.589+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94630357438976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f0dc00 con 0x5610d8e15800 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.590+0000 7ff3de854640 1 -- 172.21.15.138:0/818775627 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94630357438976] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2cfc0 con 0x5610d8e15800 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.590+0000 7ff3e1f30880 20 remove_watcher() i=2 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.590+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94630357440128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f20000 con 0x5610d8e15000 2024-04-27T05:21:57.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.591+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94630357440128] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2d200 con 0x5610d8e15000 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.591+0000 7ff3e1f30880 20 remove_watcher() i=3 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.591+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94630357441280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f20800 con 0x5610d988f000 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.592+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94630357441280] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d988f000 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.592+0000 7ff3e1f30880 20 remove_watcher() i=4 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.592+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94630357442432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f20c00 con 0x5610d8e15c00 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.593+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94630357442432] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.593+0000 7ff3e1f30880 20 remove_watcher() i=5 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.593+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94630357443584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f21000 con 0x5610d8e15c00 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.594+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94630357443584] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.594+0000 7ff3e1f30880 20 remove_watcher() i=6 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.594+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94630357444736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5610d8f21400 con 0x5610d8e15c00 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3df055640 1 -- 172.21.15.138:0/818775627 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94630357444736] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5610d8e2c900 con 0x5610d8e15c00 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 20 remove_watcher() i=7 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5610d8e15c00 msgr2=0x5610d8d7b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5610d8e15c00 0x5610d8d7b180 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5610d988f000 msgr2=0x5610d8d79b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5610d988f000 0x5610d8d79b80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5610d8e15000 msgr2=0x5610d8d7ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5610d8e15000 0x5610d8d7ac00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5610d8e15800 msgr2=0x5610d8d7a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5610d8e15800 0x5610d8d7a100 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d8e14000 msgr2=0x5610d8d79600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d8e14000 0x5610d8d79600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5610d8ba3410 tx=0x5610d99f2f60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 msgr2=0x5610d8d78b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/818775627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d988ec00 0x5610d8d78b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5610d9953950 tx=0x5610d8ba9140 comp rx=0 tx=0).stop 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 reap_dead start 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3de053640 1 -- 172.21.15.138:0/818775627 reap_dead start 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 shutdown_connections 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 >> 172.21.15.138:0/818775627 conn(0x5610d8a97400 msgr2=0x5610d8ba1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 shutdown_connections 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.595+0000 7ff3e1f30880 1 -- 172.21.15.138:0/818775627 wait complete. 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d99f5800 msgr2=0x5610d8d7b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d99f5800 0x5610d8d7b700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5610d8df7170 tx=0x5610d99f3ec0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 msgr2=0x5610d8d7a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5610d8df7830 tx=0x5610d99f3080 comp rx=0 tx=0).stop 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 shutdown_connections 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d99f5800 0x5610d8d7b700 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1160667029 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8e14c00 0x5610d8d7a680 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 >> 172.21.15.138:0/1160667029 conn(0x5610d8e3b800 msgr2=0x5610d8ba1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 shutdown_connections 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1160667029 wait complete. 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d988fc00 msgr2=0x5610d8d79080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d988fc00 0x5610d8d79080 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5610d99982d0 tx=0x5610d8ba9c20 comp rx=0 tx=0).stop 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 msgr2=0x5610d8d78580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5610d9952e10 tx=0x5610d8ba8c60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 shutdown_connections 2024-04-27T05:21:57.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5610d988fc00 0x5610d8d79080 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 --2- 172.21.15.138:0/1059610390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610d8a97000 0x5610d8d78580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 >> 172.21.15.138:0/1059610390 conn(0x5610d8a96c00 msgr2=0x5610d8b9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 shutdown_connections 2024-04-27T05:21:57.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.596+0000 7ff3e1f30880 1 -- 172.21.15.138:0/1059610390 wait complete. 2024-04-27T05:21:57.629 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:21:57.643 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:57.643 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T05:21:57.643 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T05:21:57.644 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T05:21:57.687 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:21:57.687 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a3898880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5646c188d400 0x5646c1b6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a3898880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5646c1b64600 con 0x5646c188d000 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a3898880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5646c18cc280 con 0x5646c188d400 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a183c640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5646c188d400 0x5646c1b6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a203d640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5646c188d000 0x5646c1994800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:42796/0 (socket says 172.21.15.138:42796) 2024-04-27T05:21:57.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.686+0000 7f11a203d640 1 -- 172.21.15.138:0/2242131050 learned_addr learned my addr 172.21.15.138:0/2242131050 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:21:57.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.687+0000 7f11a183c640 1 -- 172.21.15.138:0/2242131050 >> v1:172.21.15.114:6789/0 conn(0x5646c188d000 legacy=0x5646c1994800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.687+0000 7f11a183c640 1 -- 172.21.15.138:0/2242131050 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c1891380 con 0x5646c188d400 2024-04-27T05:21:57.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.687+0000 7f11a103b640 1 -- 172.21.15.138:0/2242131050 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822769891 0 0) 0x5646c18909c0 con 0x5646c188d000 2024-04-27T05:21:57.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.688+0000 7f11a183c640 1 --2- 172.21.15.138:0/2242131050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d400 0x5646c1b6e580 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5646c1998c00 tx=0x5646c199ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1d3da34cfed93a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.688+0000 7f11a103b640 1 -- 172.21.15.138:0/2242131050 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a5ce0 con 0x5646c188d400 2024-04-27T05:21:57.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.688+0000 7f11a103b640 1 -- 172.21.15.138:0/2242131050 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c1b65b00 con 0x5646c188d400 2024-04-27T05:21:57.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.688+0000 7f11a103b640 1 -- 172.21.15.138:0/2242131050 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a5e40 con 0x5646c188d400 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 -- 172.21.15.138:0/2242131050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d400 msgr2=0x5646c1b6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 --2- 172.21.15.138:0/2242131050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d400 0x5646c1b6e580 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5646c1998c00 tx=0x5646c199ede0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 -- 172.21.15.138:0/2242131050 shutdown_connections 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 --2- 172.21.15.138:0/2242131050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d400 0x5646c1b6e580 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 -- 172.21.15.138:0/2242131050 >> 172.21.15.138:0/2242131050 conn(0x5646c188cc00 msgr2=0x5646c1994000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 -- 172.21.15.138:0/2242131050 shutdown_connections 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.689+0000 7f11a3898880 1 -- 172.21.15.138:0/2242131050 wait complete. 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a3898880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a3898880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646c18cc780 con 0x5646c188d000 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a203d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a203d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:32774/0 (socket says 172.21.15.138:32774) 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a203d640 1 -- 172.21.15.138:0/1895551461 learned_addr learned my addr 172.21.15.138:0/1895551461 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a203d640 1 -- 172.21.15.138:0/1895551461 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c1890ea0 con 0x5646c188d000 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.690+0000 7f11a203d640 1 --2- 172.21.15.138:0/1895551461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x5646c2748c90 tx=0x5646c199ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f313298391cc15fb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a083a640 1 -- 172.21.15.138:0/1895551461 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a54a0 con 0x5646c188d000 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a083a640 1 -- 172.21.15.138:0/1895551461 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c1b65e00 con 0x5646c188d000 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a083a640 1 -- 172.21.15.138:0/1895551461 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a5ce0 con 0x5646c188d000 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- 172.21.15.138:0/1895551461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 msgr2=0x5646c1b6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 --2- 172.21.15.138:0/1895551461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x5646c2748c90 tx=0x5646c199ecc0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- 172.21.15.138:0/1895551461 shutdown_connections 2024-04-27T05:21:57.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 --2- 172.21.15.138:0/1895551461 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- 172.21.15.138:0/1895551461 >> 172.21.15.138:0/1895551461 conn(0x5646c188cc00 msgr2=0x5646c1995000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- 172.21.15.138:0/1895551461 shutdown_connections 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- 172.21.15.138:0/1895551461 wait complete. 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.691+0000 7f11a3898880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646c18cc8c0 con 0x5646c188d000 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a203d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a203d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:32788/0 (socket says 172.21.15.138:32788) 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a203d640 1 -- 172.21.15.138:0/342212250 learned_addr learned my addr 172.21.15.138:0/342212250 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a203d640 1 -- 172.21.15.138:0/342212250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c18916c0 con 0x5646c188d000 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a203d640 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x5646c2749350 tx=0x5646c199ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f119f037640 1 -- 172.21.15.138:0/342212250 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a5e40 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f119f037640 1 -- 172.21.15.138:0/342212250 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c275e000 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5646c18911e0 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f119f037640 1 -- 172.21.15.138:0/342212250 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a54a0 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.692+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646c18feea0 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f119f037640 1 -- 172.21.15.138:0/342212250 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5646c18b7600 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a3898880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a3898880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646c18b0280 con 0x5646c2685800 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f119f037640 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c27e0000 0x5646c1b6f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f119f037640 1 -- 172.21.15.138:0/342212250 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5646c189fa40 con 0x5646c188d000 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a283e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a283e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:32794/0 (socket says 172.21.15.138:32794) 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a203d640 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c27e0000 0x5646c1b6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a283e640 1 -- 172.21.15.138:0/3613953169 learned_addr learned my addr 172.21.15.138:0/3613953169 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a283e640 1 -- 172.21.15.138:0/3613953169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c18fed00 con 0x5646c2685800 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.693+0000 7f11a203d640 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c27e0000 0x5646c1b6f080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5646c2792420 tx=0x5646c199f4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a283e640 1 --2- 172.21.15.138:0/3613953169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x5646c2749530 tx=0x5646c199ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db0f0b0205044a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f119c832640 1 -- 172.21.15.138:0/3613953169 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a5e40 con 0x5646c2685800 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f119c832640 1 -- 172.21.15.138:0/3613953169 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c275e300 con 0x5646c2685800 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f119c832640 1 -- 172.21.15.138:0/3613953169 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a54a0 con 0x5646c2685800 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 -- 172.21.15.138:0/3613953169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 msgr2=0x5646c1b6eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 --2- 172.21.15.138:0/3613953169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x5646c2749530 tx=0x5646c199ff80 comp rx=0 tx=0).stop 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 -- 172.21.15.138:0/3613953169 shutdown_connections 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 --2- 172.21.15.138:0/3613953169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 -- 172.21.15.138:0/3613953169 >> 172.21.15.138:0/3613953169 conn(0x5646c188d400 msgr2=0x5646c1997000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 -- 172.21.15.138:0/3613953169 shutdown_connections 2024-04-27T05:21:57.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 -- 172.21.15.138:0/3613953169 wait complete. 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.694+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a3898880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a3898880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646c18b1180 con 0x5646c2685800 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a283e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a283e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:32808/0 (socket says 172.21.15.138:32808) 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 learned_addr learned my addr 172.21.15.138:0/796220615 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c18feb60 con 0x5646c2685800 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a283e640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x5646c2749920 tx=0x5646c199fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f119b02f640 1 -- 172.21.15.138:0/796220615 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a5e40 con 0x5646c2685800 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f119b02f640 1 -- 172.21.15.138:0/796220615 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c275e480 con 0x5646c2685800 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5646c18fe9c0 con 0x5646c2685800 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f119b02f640 1 -- 172.21.15.138:0/796220615 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c18a54a0 con 0x5646c2685800 2024-04-27T05:21:57.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.695+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646c18fe820 con 0x5646c2685800 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f119b02f640 1 -- 172.21.15.138:0/796220615 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5646c18b78c0 con 0x5646c2685800 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f119b02f640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c1c1e000 0x5646c1b6f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f119b02f640 1 -- 172.21.15.138:0/796220615 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5646c189fc00 con 0x5646c2685800 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f11a183c640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c1c1e000 0x5646c1b6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2685c00 0x5646c1b6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.696+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1e400 con 0x5646c2685c00 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a283e640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2685c00 0x5646c1b6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a183c640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c1c1e000 0x5646c1b6f600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5646c1998c00 tx=0x5646c280af60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a283e640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2685c00 0x5646c1b6fb80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 realm 2024-04-27T05:21:57.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1e800 con 0x5646c2685c00 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1ec00 con 0x5646c2685c00 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.697+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1f000 con 0x5646c2685c00 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1c1f400 0x5646c1b70100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1e400 con 0x5646c1c1f400 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a203d640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1c1f400 0x5646c1b70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a203d640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1c1f400 0x5646c1b70100 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 20 searching for the correct realm 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.698+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1e800 con 0x5646c1c1f400 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a203d640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1c1e800 0x5646c1b70c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1e400 con 0x5646c1c1e800 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a183c640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1c1e800 0x5646c1b70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a183c640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1c1e800 0x5646c1b70c00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a183c640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1c1e400 0x5646c1b71180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.699+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1c1e400 0x5646c1b71180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1c1e400 0x5646c1b71180 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.700+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e800 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.701+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e800 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.702+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.703+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e800 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.704+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.705+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e800 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.706+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.706+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1f400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.706+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.706+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e400 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.706+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.706+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c2685c00 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2684800 con 0x5646c1c1e800 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1ec00 con 0x5646c2685c00 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1f000 con 0x5646c2685c00 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1f800 con 0x5646c1c1f400 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 zone default found 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 4 Realm: () 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 zonegroup default 2024-04-27T05:21:57.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.707+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1fc00 con 0x5646c1c1e400 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670000 con 0x5646c1c1f400 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670400 con 0x5646c1c1f400 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:21:57.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 started sync module instance, tier type = 2024-04-27T05:21:57.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:21:57.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.708+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670800 con 0x5646c1c1e800 2024-04-27T05:21:57.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.709+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.709+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94861897656576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670c00 con 0x5646c1c1e800 2024-04-27T05:21:57.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.709+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2671000 con 0x5646c2685c00 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.710+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94861897656576] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.710+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.710+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94861897657728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2671400 con 0x5646c2685c00 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.710+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2671800 con 0x5646c1c1f400 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.712+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.712+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94861897657728] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.712+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94861897658880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670000 con 0x5646c1c1f400 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.712+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670400 con 0x5646c1c1e800 2024-04-27T05:21:57.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.713+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94861897658880] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.713+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.713+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94861897660032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2671800 con 0x5646c1c1e800 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.713+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1f800 con 0x5646c2685c00 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.714+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94861897660032] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.714+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.714+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94861897661184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1ec00 con 0x5646c2685c00 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.714+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1f000 con 0x5646c1c1e400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.715+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94861897661184] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.715+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.715+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94861897662336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1fc00 con 0x5646c1c1e400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.715+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c27e0c00 con 0x5646c1c1e400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.716+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94861897662336] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.716+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.716+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94861897663488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670800 con 0x5646c1c1e400 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.716+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670c00 con 0x5646c1c1e400 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94861897664640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670400 con 0x5646c1c1e400 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94861897663488] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=0 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=1 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=2 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=3 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=4 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=5 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.717+0000 7f11a3898880 20 add_watcher() i=6 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.718+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94861897664640] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.718+0000 7f11a3898880 20 add_watcher() i=7 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.718+0000 7f11a3898880 2 all 8 watchers are set, enabling cache 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.718+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2671800 con 0x5646c2685c00 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.718+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.718+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94861897665792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2671000 con 0x5646c2685c00 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94861897665792] v31'197 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f118900b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a3898880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a3898880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a3898880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.719+0000 7f11a3898880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646c18b0780 con 0x5646c2671c00 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f11a183c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f11a183c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:32824/0 (socket says 172.21.15.138:32824) 2024-04-27T05:21:57.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f11a183c640 1 -- 172.21.15.138:0/3821123498 learned_addr learned my addr 172.21.15.138:0/3821123498 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f11a183c640 1 -- 172.21.15.138:0/3821123498 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c18fe680 con 0x5646c2671c00 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f11a183c640 1 --2- 172.21.15.138:0/3821123498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5646c1bfec30 tx=0x5646c280afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1da94f58386810 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f118880a640 1 -- 172.21.15.138:0/3821123498 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c2672580 con 0x5646c2671c00 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f118880a640 1 -- 172.21.15.138:0/3821123498 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c275e000 con 0x5646c2671c00 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.720+0000 7f118880a640 1 -- 172.21.15.138:0/3821123498 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c26726e0 con 0x5646c2671c00 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- 172.21.15.138:0/3821123498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 msgr2=0x5646c1b70680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 --2- 172.21.15.138:0/3821123498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5646c1bfec30 tx=0x5646c280afc0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- 172.21.15.138:0/3821123498 shutdown_connections 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 --2- 172.21.15.138:0/3821123498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- 172.21.15.138:0/3821123498 >> 172.21.15.138:0/3821123498 conn(0x5646c2671400 msgr2=0x5646c1997800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- 172.21.15.138:0/3821123498 shutdown_connections 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- 172.21.15.138:0/3821123498 wait complete. 2024-04-27T05:21:57.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 Processor -- start 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- start start 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.721+0000 7f11a3898880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646c18b0500 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a183c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a183c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:32834/0 (socket says 172.21.15.138:32834) 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 learned_addr learned my addr 172.21.15.138:0/3883637917 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646c1890ea0 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a183c640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x5646c1bff7a0 tx=0x5646c280ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f1187007640 1 -- 172.21.15.138:0/3883637917 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c2672840 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f1187007640 1 -- 172.21.15.138:0/3883637917 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646c275e780 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5646c18916c0 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f1187007640 1 -- 172.21.15.138:0/3883637917 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646c2672580 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.722+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646c18911e0 con 0x5646c2671c00 2024-04-27T05:21:57.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f1187007640 1 -- 172.21.15.138:0/3883637917 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5646c18b6b00 con 0x5646c2671c00 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f1187007640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c2670800 0x5646c1b71700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f1187007640 1 -- 172.21.15.138:0/3883637917 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5646c189fdc0 con 0x5646c2671c00 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f11a283e640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c2670800 0x5646c1b71700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2684800 0x5646c1cce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2000 con 0x5646c2684800 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f11a203d640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2684800 0x5646c1cce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.723+0000 7f11a283e640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c2670800 0x5646c1b71700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5646c280ca20 tx=0x5646c1cd4000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.724+0000 7f11a203d640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2684800 0x5646c1cce000 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.724+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'59 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c2684800 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.725+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1cd2400 0x5646c1cce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.725+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2800 con 0x5646c1cd2400 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.725+0000 7f11a183c640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1cd2400 0x5646c1cce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.725+0000 7f11a183c640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1cd2400 0x5646c1cce580 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.726+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.726+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1cd2c00 0x5646c1cceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.726+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3000 con 0x5646c1cd2c00 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.726+0000 7f11a283e640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1cd2c00 0x5646c1cceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.726+0000 7f11a283e640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1cd2c00 0x5646c1cceb00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.727+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.727+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3400 con 0x5646c2684800 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.728+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'198 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c2684800 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.728+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1cd3800 0x5646c1ccf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.728+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3c00 con 0x5646c1cd3800 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.728+0000 7f11a203d640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1cd3800 0x5646c1ccf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.728+0000 7f11a203d640 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1cd3800 0x5646c1ccf080 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.729+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'71 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.729+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8000 con 0x5646c1cd2c00 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.730+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'58 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.730+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8400 con 0x5646c2684800 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.732+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'34 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c2684800 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.732+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8800 con 0x5646c1cd2400 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.733+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'49 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.733+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8c00 con 0x5646c1cd3800 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.734+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.734+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9000 con 0x5646c1cd2c00 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.734+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.734+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9400 con 0x5646c1cd3800 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.735+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'72 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.735+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9800 con 0x5646c1cd2400 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.737+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'31 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.737+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9c00 con 0x5646c1cd3800 2024-04-27T05:21:57.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.737+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.737+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8400 con 0x5646c1cd2400 2024-04-27T05:21:57.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.738+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'66 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.738+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8c00 con 0x5646c2684800 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.739+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'199 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c2684800 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.740+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2000 con 0x5646c1cd2c00 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.740+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'71 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.741+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3400 con 0x5646c1cd3800 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.741+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'73 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.741+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3c00 con 0x5646c1cd2c00 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.742+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'37 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.742+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670000 con 0x5646c2684800 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.743+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c2684800 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.743+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3000 con 0x5646c2684800 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.744+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'200 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c2684800 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.744+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2000 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.745+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'72 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.745+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3c00 con 0x5646c1cd3800 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.746+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'41 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.746+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8000 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.747+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.747+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9000 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.748+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'43 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.748+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670c00 con 0x5646c1cd2c00 2024-04-27T05:21:57.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.748+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.748+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670400 con 0x5646c1cd2c00 2024-04-27T05:21:57.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.749+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'59 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.749+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3400 con 0x5646c1cd2400 2024-04-27T05:21:57.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.750+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'34 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.750+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3000 con 0x5646c1cd2c00 2024-04-27T05:21:57.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.751+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'37 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.751+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3c00 con 0x5646c1cd2400 2024-04-27T05:21:57.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.752+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'50 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.752+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670000 con 0x5646c1cd2c00 2024-04-27T05:21:57.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.753+0000 7f11a283e640 1 -- 172.21.15.138:0/3883637917 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'73 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1cd2c00 2024-04-27T05:21:57.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.753+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9400 con 0x5646c1cd2400 2024-04-27T05:21:57.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.754+0000 7f11a183c640 1 -- 172.21.15.138:0/3883637917 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'67 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1cd2400 2024-04-27T05:21:57.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.754+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9c00 con 0x5646c1cd3800 2024-04-27T05:21:57.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.755+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1cd3800 2024-04-27T05:21:57.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.755+0000 7f1184802640 20 reqs_thread_entry: start 2024-04-27T05:21:57.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 20 init_complete bucket index max shards: 11 2024-04-27T05:21:57.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:21:57.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11827fe640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:22:28 2024 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f1181fdc640 20 reqs_thread_entry: start 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 20 rados->read ofs=0 len=0 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2800 con 0x5646c2685c00 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.756+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3400 con 0x5646c1c1e800 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.757+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv40 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.759 INFO:teuthology.orchestra.run.smithi138.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T05:21:57.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.757+0000 7f118900b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:21:57.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.757+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94861897665792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8800 con 0x5646c2685c00 2024-04-27T05:21:57.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.758+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94861897665792] v31'201 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.758+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94861897656576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf9800 con 0x5646c1c1e800 2024-04-27T05:21:57.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.759+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94861897656576] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.759+0000 7f11a3898880 20 remove_watcher() i=0 2024-04-27T05:21:57.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.759+0000 7f11a3898880 2 removed watcher, disabling cache 2024-04-27T05:21:57.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.759+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94861897657728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cf8400 con 0x5646c2685c00 2024-04-27T05:21:57.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.760+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94861897657728] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.760+0000 7f11a3898880 20 remove_watcher() i=1 2024-04-27T05:21:57.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.760+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94861897658880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1fc00 con 0x5646c1c1f400 2024-04-27T05:21:57.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.761+0000 7f11a203d640 1 -- 172.21.15.138:0/796220615 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94861897658880] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30d80 con 0x5646c1c1f400 2024-04-27T05:21:57.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.761+0000 7f11a3898880 20 remove_watcher() i=2 2024-04-27T05:21:57.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.761+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94861897660032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1c1f000 con 0x5646c1c1e800 2024-04-27T05:21:57.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.762+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94861897660032] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c30fc0 con 0x5646c1c1e800 2024-04-27T05:21:57.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.762+0000 7f11a3898880 20 remove_watcher() i=3 2024-04-27T05:21:57.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.762+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94861897661184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2000 con 0x5646c2685c00 2024-04-27T05:21:57.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.763+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94861897661184] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c2685c00 2024-04-27T05:21:57.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.763+0000 7f11a3898880 20 remove_watcher() i=4 2024-04-27T05:21:57.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.763+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94861897662336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd3000 con 0x5646c1c1e400 2024-04-27T05:21:57.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.764+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94861897662336] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.764+0000 7f11a3898880 20 remove_watcher() i=5 2024-04-27T05:21:57.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.764+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94861897663488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c1cd2800 con 0x5646c1c1e400 2024-04-27T05:21:57.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.765+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94861897663488] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.765+0000 7f11a3898880 20 remove_watcher() i=6 2024-04-27T05:21:57.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.765+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94861897664640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646c2670c00 con 0x5646c1c1e400 2024-04-27T05:21:57.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a283e640 1 -- 172.21.15.138:0/796220615 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94861897664640] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5646c1c306c0 con 0x5646c1c1e400 2024-04-27T05:21:57.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 20 remove_watcher() i=7 2024-04-27T05:21:57.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1c1e400 msgr2=0x5646c1b71180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1c1e400 0x5646c1b71180 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2685c00 msgr2=0x5646c1b6fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2685c00 0x5646c1b6fb80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1c1e800 msgr2=0x5646c1b70c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1c1e800 0x5646c1b70c00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1c1f400 msgr2=0x5646c1b70100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1c1f400 0x5646c1b70100 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c1c1e000 msgr2=0x5646c1b6f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c1c1e000 0x5646c1b6f600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5646c1998c00 tx=0x5646c280af60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 msgr2=0x5646c1b6eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 --2- 172.21.15.138:0/796220615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2685800 0x5646c1b6eb00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x5646c2749920 tx=0x5646c199fbc0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 reap_dead start 2024-04-27T05:21:57.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a183c640 1 -- 172.21.15.138:0/796220615 reap_dead start 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 shutdown_connections 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.766+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 >> 172.21.15.138:0/796220615 conn(0x5646c188d400 msgr2=0x5646c1997000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 shutdown_connections 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/796220615 wait complete. 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1cd2400 msgr2=0x5646c1cce580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5646c1cd2400 0x5646c1cce580 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2684800 msgr2=0x5646c1cce000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5646c2684800 0x5646c1cce000 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1cd3800 msgr2=0x5646c1ccf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5646c1cd3800 0x5646c1ccf080 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1cd2c00 msgr2=0x5646c1cceb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5646c1cd2c00 0x5646c1cceb00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c2670800 msgr2=0x5646c1b71700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c2670800 0x5646c1b71700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5646c280ca20 tx=0x5646c1cd4000 comp rx=0 tx=0).stop 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 msgr2=0x5646c1b70680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x5646c1bff7a0 tx=0x5646c280ae40 comp rx=0 tx=0).stop 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a203d640 1 -- 172.21.15.138:0/3883637917 reap_dead start 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 shutdown_connections 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/3883637917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c2671c00 0x5646c1b70680 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 >> 172.21.15.138:0/3883637917 conn(0x5646c2671400 msgr2=0x5646c1997800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 shutdown_connections 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/3883637917 wait complete. 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c27e0000 msgr2=0x5646c1b6f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c27e0000 0x5646c1b6f080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5646c2792420 tx=0x5646c199f4a0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 msgr2=0x5646c1b6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x5646c2749350 tx=0x5646c199ec60 comp rx=0 tx=0).stop 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 shutdown_connections 2024-04-27T05:21:57.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5646c27e0000 0x5646c1b6f080 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 --2- 172.21.15.138:0/342212250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5646c188d000 0x5646c1b6e580 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:21:57.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.767+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 >> 172.21.15.138:0/342212250 conn(0x5646c188cc00 msgr2=0x5646c1995000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:21:57.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.768+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 shutdown_connections 2024-04-27T05:21:57.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:21:57.768+0000 7f11a3898880 1 -- 172.21.15.138:0/342212250 wait complete. 2024-04-27T05:21:57.774 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T05:21:57.778 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:21:57.785 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-27T05:22:32.786 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-04-27T05:22:32.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-04-27T05:22:32.787 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-04-27T05:22:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.835+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.835+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3e43f880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5587929fd000 0x558792cde580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3e43f880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558792cd4600 con 0x5587929fd400 2024-04-27T05:22:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3e43f880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x558792a3c280 con 0x5587929fd000 2024-04-27T05:22:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3cbeb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5587929fd000 0x558792cde580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3c3ea640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5587929fd400 0x558792b04800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40886/0 (socket says 172.21.15.138:40886) 2024-04-27T05:22:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3c3ea640 1 -- 172.21.15.138:0/3229628992 learned_addr learned my addr 172.21.15.138:0/3229628992 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3bbe9640 1 -- 172.21.15.138:0/3229628992 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057062271 0 0) 0x558792a009c0 con 0x5587929fd400 2024-04-27T05:22:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.836+0000 7efe3bbe9640 1 -- 172.21.15.138:0/3229628992 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x558792cd5980 con 0x5587929fd400 2024-04-27T05:22:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.837+0000 7efe3cbeb640 1 -- 172.21.15.138:0/3229628992 >> v1:172.21.15.114:6789/0 conn(0x5587929fd400 legacy=0x558792b04800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.837+0000 7efe3cbeb640 1 -- 172.21.15.138:0/3229628992 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a01380 con 0x5587929fd000 2024-04-27T05:22:32.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.838+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/3229628992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x558792b08d50 tx=0x558792b0ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebaed83e99698b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.838+0000 7efe3bbe9640 1 -- 172.21.15.138:0/3229628992 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a15ce0 con 0x5587929fd000 2024-04-27T05:22:32.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3bbe9640 1 -- 172.21.15.138:0/3229628992 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558792cd5980 con 0x5587929fd000 2024-04-27T05:22:32.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3bbe9640 1 -- 172.21.15.138:0/3229628992 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a15e40 con 0x5587929fd000 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 -- 172.21.15.138:0/3229628992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 msgr2=0x558792cde580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 --2- 172.21.15.138:0/3229628992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x558792b08d50 tx=0x558792b0ee40 comp rx=0 tx=0).stop 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 -- 172.21.15.138:0/3229628992 shutdown_connections 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 --2- 172.21.15.138:0/3229628992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 -- 172.21.15.138:0/3229628992 >> 172.21.15.138:0/3229628992 conn(0x5587929fcc00 msgr2=0x558792b04000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 -- 172.21.15.138:0/3229628992 shutdown_connections 2024-04-27T05:22:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.839+0000 7efe3e43f880 1 -- 172.21.15.138:0/3229628992 wait complete. 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.840+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.840+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.840+0000 7efe3e43f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.840+0000 7efe3e43f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558792a3c780 con 0x5587929fd000 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3cbeb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3cbeb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35654/0 (socket says 172.21.15.138:35654) 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1581622914 learned_addr learned my addr 172.21.15.138:0/1581622914 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1581622914 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a00ea0 con 0x5587929fd000 2024-04-27T05:22:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/1581622914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x558793803590 tx=0x558792b0eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db60115f92b54f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3b3e8640 1 -- 172.21.15.138:0/1581622914 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a154a0 con 0x5587929fd000 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3b3e8640 1 -- 172.21.15.138:0/1581622914 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558792cd5c80 con 0x5587929fd000 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.841+0000 7efe3b3e8640 1 -- 172.21.15.138:0/1581622914 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a15ce0 con 0x5587929fd000 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- 172.21.15.138:0/1581622914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 msgr2=0x558792cde580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1581622914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x558793803590 tx=0x558792b0eea0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- 172.21.15.138:0/1581622914 shutdown_connections 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1581622914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- 172.21.15.138:0/1581622914 >> 172.21.15.138:0/1581622914 conn(0x5587929fcc00 msgr2=0x558792b05000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- 172.21.15.138:0/1581622914 shutdown_connections 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- 172.21.15.138:0/1581622914 wait complete. 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.842+0000 7efe3e43f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558792a3c8c0 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3cbeb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3cbeb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35666/0 (socket says 172.21.15.138:35666) 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3cbeb640 1 -- 172.21.15.138:0/2030537746 learned_addr learned my addr 172.21.15.138:0/2030537746 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3cbeb640 1 -- 172.21.15.138:0/2030537746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a016c0 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x5587938b4e10 tx=0x558792b0ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe39be5640 1 -- 172.21.15.138:0/2030537746 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a15e40 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe39be5640 1 -- 172.21.15.138:0/2030537746 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558792cd5e00 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558792a011e0 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe39be5640 1 -- 172.21.15.138:0/2030537746 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a154a0 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.843+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558792a6eea0 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe39be5640 1 -- 172.21.15.138:0/2030537746 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558792a26b00 con 0x5587929fd000 2024-04-27T05:22:32.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe3e43f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe3e43f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558792a3cc80 con 0x5587937f4c00 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe39be5640 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5587937f5c00 0x558792cdf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe39be5640 1 -- 172.21.15.138:0/2030537746 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x558792a0fa40 con 0x5587929fd000 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.844+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5587937f5c00 0x558792cdf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe3d3ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe3d3ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35670/0 (socket says 172.21.15.138:35670) 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe3d3ec640 1 -- 172.21.15.138:0/562371440 learned_addr learned my addr 172.21.15.138:0/562371440 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5587937f5c00 0x558792cdf080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5587938f62d0 tx=0x558792b0fb00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe3d3ec640 1 -- 172.21.15.138:0/562371440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a6ed00 con 0x5587937f4c00 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/562371440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x5587938b55c0 tx=0x558792b0ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322b34159c24c579 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe373e0640 1 -- 172.21.15.138:0/562371440 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a15e40 con 0x5587937f4c00 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe373e0640 1 -- 172.21.15.138:0/562371440 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558793956000 con 0x5587937f4c00 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.845+0000 7efe373e0640 1 -- 172.21.15.138:0/562371440 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a154a0 con 0x5587937f4c00 2024-04-27T05:22:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- 172.21.15.138:0/562371440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 msgr2=0x558792cdeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 --2- 172.21.15.138:0/562371440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x5587938b55c0 tx=0x558792b0ff80 comp rx=0 tx=0).stop 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- 172.21.15.138:0/562371440 shutdown_connections 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 --2- 172.21.15.138:0/562371440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- 172.21.15.138:0/562371440 >> 172.21.15.138:0/562371440 conn(0x5587929fd400 msgr2=0x558792b07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- 172.21.15.138:0/562371440 shutdown_connections 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- 172.21.15.138:0/562371440 wait complete. 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3e43f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558792a20280 con 0x5587937f4c00 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3d3ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3d3ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35674/0 (socket says 172.21.15.138:35674) 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.846+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 learned_addr learned my addr 172.21.15.138:0/1111845265 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a6eb60 con 0x5587937f4c00 2024-04-27T05:22:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x5587938b5980 tx=0x558792b0fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe35bdd640 1 -- 172.21.15.138:0/1111845265 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a15e40 con 0x5587937f4c00 2024-04-27T05:22:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe35bdd640 1 -- 172.21.15.138:0/1111845265 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558793956180 con 0x5587937f4c00 2024-04-27T05:22:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558792a6e9c0 con 0x5587937f4c00 2024-04-27T05:22:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe35bdd640 1 -- 172.21.15.138:0/1111845265 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558792a154a0 con 0x5587937f4c00 2024-04-27T05:22:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.847+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558792a6e820 con 0x5587937f4c00 2024-04-27T05:22:32.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe35bdd640 1 -- 172.21.15.138:0/1111845265 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558792a27600 con 0x5587937f4c00 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe35bdd640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558792d76000 0x558792cdf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe35bdd640 1 -- 172.21.15.138:0/1111845265 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x558792a0fc00 con 0x5587937f4c00 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558792d76000 0x558792cdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5587937f5000 0x558792cdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587937f5400 con 0x5587937f5000 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.848+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5587937f5000 0x558792cdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558792d76000 0x558792cdf600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x558792b09410 tx=0x55879396aea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5587937f5000 0x558792cdfb80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 realm 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587937f5800 con 0x5587937f5000 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77000 con 0x5587937f5000 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.849+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77400 con 0x5587937f5000 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3e43f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558792d77800 0x558792ce0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77c00 con 0x558792d77800 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558792d77800 0x558792ce0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558792d77800 0x558792ce0100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.850+0000 7efe3e43f880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3e43f880 20 searching for the correct realm 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77000 con 0x558792d77800 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3cbeb640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558792d77000 0x558792ce0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77c00 con 0x558792d77000 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558792d77000 0x558792ce0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.851+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558792d77000 0x558792ce0c00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.852+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.852+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558792d77c00 0x558792ce1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.852+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.852+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558792d77c00 0x558792ce1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.852+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558792d77c00 0x558792ce1180 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.853+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77000 2024-04-27T05:22:32.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.854+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.855+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.855+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.855+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.855+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.855+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.855+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.856+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.857+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.858+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.858+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77000 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.858+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.858+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.858+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.858+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77000 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.859+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77800 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.860+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.860+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77c00 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.860+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.860+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x5587937f5000 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d76c00 con 0x558792d77000 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:32.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77400 con 0x5587937f5000 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587937f5400 con 0x5587937f5000 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.861+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587937f5800 con 0x558792d77800 2024-04-27T05:22:32.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 zone default found 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 4 Realm: () 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 zonegroup default 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2000 con 0x558792d77c00 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2400 con 0x558792d77800 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.862+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3e43f880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2800 con 0x558792d77800 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3e43f880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3e43f880 20 started sync module instance, tier type = 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3e43f880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.863+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2c00 con 0x558792d77000 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.864+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.864+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94040771881216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3000 con 0x558792d77000 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.864+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3400 con 0x5587937f5000 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.865+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94040771881216] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.865+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.865+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94040771882368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3800 con 0x5587937f5000 2024-04-27T05:22:32.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.865+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3c00 con 0x558792d77800 2024-04-27T05:22:32.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.867+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.867+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94040771883520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2400 con 0x558792d77800 2024-04-27T05:22:32.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.867+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2800 con 0x558792d77000 2024-04-27T05:22:32.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.867+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94040771882368] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.868+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94040771883520] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.868+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.868+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94040771884672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3c00 con 0x558792d77000 2024-04-27T05:22:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.868+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2000 con 0x5587937f5000 2024-04-27T05:22:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.869+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.869+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94040771885824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587937f5800 con 0x5587937f5000 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.869+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587937f5400 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.869+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94040771884672] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.870+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.870+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94040771885824] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.870+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94040771886976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792d77400 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.870+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558793950c00 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.871+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94040771886976] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.871+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.871+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94040771888128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558793950800 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.871+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2c00 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94040771888128] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94040771889280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3000 con 0x558792d77c00 2024-04-27T05:22:32.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=0 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=1 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=2 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=3 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=4 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=5 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.874+0000 7efe3e43f880 20 add_watcher() i=6 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.875+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94040771889280] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.875+0000 7efe3e43f880 20 add_watcher() i=7 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.875+0000 7efe3e43f880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.875+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2800 con 0x5587937f5000 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.875+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.875+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94040771890432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3400 con 0x5587937f5000 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94040771890432] v31'202 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe23bb9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3e43f880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3e43f880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3e43f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3e43f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558792a208c0 con 0x558792d76c00 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3c3ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3c3ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35682/0 (socket says 172.21.15.138:35682) 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.878+0000 7efe3c3ea640 1 -- 172.21.15.138:0/4108824466 learned_addr learned my addr 172.21.15.138:0/4108824466 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.879+0000 7efe3c3ea640 1 -- 172.21.15.138:0/4108824466 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a6e680 con 0x558792d76c00 2024-04-27T05:22:32.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.879+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/4108824466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x558792d5b620 tx=0x55879396ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e35349e9565d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.879+0000 7efe233b8640 1 -- 172.21.15.138:0/4108824466 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5587938d4580 con 0x558792d76c00 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.879+0000 7efe233b8640 1 -- 172.21.15.138:0/4108824466 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558793956a80 con 0x558792d76c00 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.879+0000 7efe233b8640 1 -- 172.21.15.138:0/4108824466 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5587938d46e0 con 0x558792d76c00 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- 172.21.15.138:0/4108824466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 msgr2=0x558792ce0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 --2- 172.21.15.138:0/4108824466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x558792d5b620 tx=0x55879396ae40 comp rx=0 tx=0).stop 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- 172.21.15.138:0/4108824466 shutdown_connections 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 --2- 172.21.15.138:0/4108824466 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- 172.21.15.138:0/4108824466 >> 172.21.15.138:0/4108824466 conn(0x5587938d3800 msgr2=0x558792b07800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- 172.21.15.138:0/4108824466 shutdown_connections 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- 172.21.15.138:0/4108824466 wait complete. 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 Processor -- start 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- start start 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3e43f880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558792a20780 con 0x558792d76c00 2024-04-27T05:22:32.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.880+0000 7efe3c3ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe3c3ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35686/0 (socket says 172.21.15.138:35686) 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe3c3ea640 1 -- 172.21.15.138:0/971401479 learned_addr learned my addr 172.21.15.138:0/971401479 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe3c3ea640 1 -- 172.21.15.138:0/971401479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558792a6e4e0 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe3c3ea640 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x558792d5b830 tx=0x55879396afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe21bb5640 1 -- 172.21.15.138:0/971401479 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5587938d4840 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe21bb5640 1 -- 172.21.15.138:0/971401479 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558793956c00 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558792a01380 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe21bb5640 1 -- 172.21.15.138:0/971401479 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5587938d4580 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.881+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558792a00ea0 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe21bb5640 1 -- 172.21.15.138:0/971401479 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558792a27b80 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe21bb5640 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558793950c00 0x558792ce1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe21bb5640 1 -- 172.21.15.138:0/971401479 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x558792a0fdc0 con 0x558792d76c00 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe3e43f880 5 note: GC not initialized 2024-04-27T05:22:32.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558793950c00 0x558792ce1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe1f3b0640 20 reqs_thread_entry: start 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.882+0000 7efe3e43f880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3d3ec640 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558793950c00 0x558792ce1700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5587939626c0 tx=0x55879396bda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3e43f880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe1d3ac640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe1cb8a640 20 reqs_thread_entry: start 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a000 con 0x5587937f5000 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a400 con 0x558792d77000 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.883+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a800 con 0x5587937f5000 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7ac00 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b000 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b400 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b800 con 0x5587937f5000 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.884+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7bc00 con 0x5587937f5000 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.885+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.885+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6000 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.885+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.885+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6400 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.885+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.885+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6800 con 0x558792d77c00 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6c00 con 0x5587937f5000 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7000 con 0x558792d77000 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7400 con 0x558792d77800 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.886+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7800 con 0x558792d77c00 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.887+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.887+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7c00 con 0x558792d77c00 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.887+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.887+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6000 con 0x558792d77800 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.887+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.887+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6400 con 0x558792d77c00 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7400 con 0x5587937f5000 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6800 con 0x558792d77800 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7ac00 con 0x558792d77c00 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.888+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b000 con 0x558792d77000 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b400 con 0x558792d77c00 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a000 con 0x5587937f5000 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a800 con 0x5587937f5000 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.889+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b800 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7bc00 con 0x5587937f5000 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2400 con 0x5587937f5000 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d2c00 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.890+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938d3000 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.891+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.891+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7ac00 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.891+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.891+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7b400 con 0x558792d77c00 2024-04-27T05:22:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.891+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.892+0000 7efe23bb9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.892+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94040771890432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a000 con 0x5587937f5000 2024-04-27T05:22:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.893+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94040771890432] v31'203 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.893+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94040771881216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6c00 con 0x558792d77000 2024-04-27T05:22:32.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.894+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94040771881216] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.894+0000 7efe3e43f880 20 remove_watcher() i=0 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.894+0000 7efe3e43f880 2 removed watcher, disabling cache 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.894+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94040771882368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7800 con 0x5587937f5000 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.895+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94040771882368] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.895+0000 7efe3e43f880 20 remove_watcher() i=1 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.895+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94040771883520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7c00 con 0x558792d77800 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.897+0000 7efe3cbeb640 1 -- 172.21.15.138:0/1111845265 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94040771883520] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94fc0 con 0x558792d77800 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.897+0000 7efe3e43f880 20 remove_watcher() i=2 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.897+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94040771884672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6400 con 0x558792d77000 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.898+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94040771884672] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d95200 con 0x558792d77000 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.898+0000 7efe3e43f880 20 remove_watcher() i=3 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.898+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94040771885824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7400 con 0x5587937f5000 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.899+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94040771885824] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x5587937f5000 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.899+0000 7efe3e43f880 20 remove_watcher() i=4 2024-04-27T05:22:32.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.899+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94040771886976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b7000 con 0x558792d77c00 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.900+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94040771886976] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.900+0000 7efe3e43f880 20 remove_watcher() i=5 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.900+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94040771888128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5587938b6c00 con 0x558792d77c00 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.901+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94040771888128] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.901+0000 7efe3e43f880 20 remove_watcher() i=6 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.901+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94040771889280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558792e7a400 con 0x558792d77c00 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3d3ec640 1 -- 172.21.15.138:0/1111845265 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94040771889280] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558792d94900 con 0x558792d77c00 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 20 remove_watcher() i=7 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558792d77c00 msgr2=0x558792ce1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558792d77c00 0x558792ce1180 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5587937f5000 msgr2=0x558792cdfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5587937f5000 0x558792cdfb80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558792d77000 msgr2=0x558792ce0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558792d77000 0x558792ce0c00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558792d77800 msgr2=0x558792ce0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558792d77800 0x558792ce0100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558792d76000 msgr2=0x558792cdf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558792d76000 0x558792cdf600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x558792b09410 tx=0x55879396aea0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 msgr2=0x558792cdeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/1111845265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587937f4c00 0x558792cdeb00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x5587938b5980 tx=0x558792b0fec0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 reap_dead start 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 shutdown_connections 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3c3ea640 1 -- 172.21.15.138:0/1111845265 reap_dead start 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 >> 172.21.15.138:0/1111845265 conn(0x5587929fd400 msgr2=0x558792b07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 shutdown_connections 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/1111845265 wait complete. 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558793950c00 msgr2=0x558792ce1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558793950c00 0x558792ce1700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5587939626c0 tx=0x55879396bda0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 msgr2=0x558792ce0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x558792d5b830 tx=0x55879396afc0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 shutdown_connections 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.902+0000 7efe3e43f880 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558793950c00 0x558792ce1700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 --2- 172.21.15.138:0/971401479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558792d76c00 0x558792ce0680 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 >> 172.21.15.138:0/971401479 conn(0x5587938d3800 msgr2=0x558792b07800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 shutdown_connections 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/971401479 wait complete. 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5587937f5c00 msgr2=0x558792cdf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5587937f5c00 0x558792cdf080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5587938f62d0 tx=0x558792b0fb00 comp rx=0 tx=0).stop 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 msgr2=0x558792cde580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x5587938b4e10 tx=0x558792b0ed20 comp rx=0 tx=0).stop 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 shutdown_connections 2024-04-27T05:22:32.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5587937f5c00 0x558792cdf080 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 --2- 172.21.15.138:0/2030537746 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587929fd000 0x558792cde580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 >> 172.21.15.138:0/2030537746 conn(0x5587929fcc00 msgr2=0x558792b05000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 shutdown_connections 2024-04-27T05:22:32.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.903+0000 7efe3e43f880 1 -- 172.21.15.138:0/2030537746 wait complete. 2024-04-27T05:22:32.915 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "anonymous", 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "anonymous", 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:32.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:32.917 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:32.918 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:32.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "anonymous", 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [], 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:32.920 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 0, 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 0 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:32.922 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:32.922 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'anonymous', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T05:22:32.923 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-27T05:22:32.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-27T05:22:32.923 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-27T05:22:32.964 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:32.964 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:32.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.962+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:32.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.962+0000 7fa354215880 1 -- start start 2024-04-27T05:22:32.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.962+0000 7fa354215880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x556d7a7b1400 0x556d7aa92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.962+0000 7fa354215880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556d7aa88600 con 0x556d7a7b1000 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.962+0000 7fa354215880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x556d7a7f0280 con 0x556d7a7b1400 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa34fe88640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x556d7a7b1400 0x556d7aa92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa350689640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x556d7a7b1000 0x556d7a8b8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40900/0 (socket says 172.21.15.138:40900) 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa350689640 1 -- 172.21.15.138:0/3556883789 learned_addr learned my addr 172.21.15.138:0/3556883789 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa34f687640 1 -- 172.21.15.138:0/3556883789 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70392878 0 0) 0x556d7a7b49c0 con 0x556d7a7b1000 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa34f687640 1 -- 172.21.15.138:0/3556883789 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x556d7aa89980 con 0x556d7a7b1000 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa34fe88640 1 -- 172.21.15.138:0/3556883789 >> v1:172.21.15.114:6789/0 conn(0x556d7a7b1000 legacy=0x556d7a8b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.963+0000 7fa34fe88640 1 -- 172.21.15.138:0/3556883789 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a7b5380 con 0x556d7a7b1400 2024-04-27T05:22:32.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa34fe88640 1 --2- 172.21.15.138:0/3556883789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1400 0x556d7aa92580 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x556d7a8bcc00 tx=0x556d7a8c2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2b146fbbee430 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa34f687640 1 -- 172.21.15.138:0/3556883789 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c9ce0 con 0x556d7a7b1400 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa34f687640 1 -- 172.21.15.138:0/3556883789 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7aa89980 con 0x556d7a7b1400 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa34f687640 1 -- 172.21.15.138:0/3556883789 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c9e40 con 0x556d7a7b1400 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 -- 172.21.15.138:0/3556883789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1400 msgr2=0x556d7aa92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 --2- 172.21.15.138:0/3556883789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1400 0x556d7aa92580 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x556d7a8bcc00 tx=0x556d7a8c2e40 comp rx=0 tx=0).stop 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 -- 172.21.15.138:0/3556883789 shutdown_connections 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 --2- 172.21.15.138:0/3556883789 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1400 0x556d7aa92580 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 -- 172.21.15.138:0/3556883789 >> 172.21.15.138:0/3556883789 conn(0x556d7a7b0c00 msgr2=0x556d7a8b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 -- 172.21.15.138:0/3556883789 shutdown_connections 2024-04-27T05:22:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.965+0000 7fa354215880 1 -- 172.21.15.138:0/3556883789 wait complete. 2024-04-27T05:22:32.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa354215880 1 -- start start 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa354215880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa354215880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7a7f0780 con 0x556d7a7b1000 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa350689640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa350689640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35702/0 (socket says 172.21.15.138:35702) 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.966+0000 7fa350689640 1 -- 172.21.15.138:0/2332117941 learned_addr learned my addr 172.21.15.138:0/2332117941 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa350689640 1 -- 172.21.15.138:0/2332117941 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a7b4ea0 con 0x556d7a7b1000 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa350689640 1 --2- 172.21.15.138:0/2332117941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x556d7b668c90 tx=0x556d7a8c2ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43fc48dfc06e540 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa34ee86640 1 -- 172.21.15.138:0/2332117941 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c94a0 con 0x556d7a7b1000 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa34ee86640 1 -- 172.21.15.138:0/2332117941 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7aa89e00 con 0x556d7a7b1000 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa34ee86640 1 -- 172.21.15.138:0/2332117941 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c9ce0 con 0x556d7a7b1000 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 -- 172.21.15.138:0/2332117941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 msgr2=0x556d7aa92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 --2- 172.21.15.138:0/2332117941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x556d7b668c90 tx=0x556d7a8c2ea0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 -- 172.21.15.138:0/2332117941 shutdown_connections 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 --2- 172.21.15.138:0/2332117941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 -- 172.21.15.138:0/2332117941 >> 172.21.15.138:0/2332117941 conn(0x556d7a7b0c00 msgr2=0x556d7a8b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 -- 172.21.15.138:0/2332117941 shutdown_connections 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.967+0000 7fa354215880 1 -- 172.21.15.138:0/2332117941 wait complete. 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa354215880 1 -- start start 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa354215880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa354215880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7a7f08c0 con 0x556d7a7b1000 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa350689640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa350689640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35718/0 (socket says 172.21.15.138:35718) 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa350689640 1 -- 172.21.15.138:0/1920846699 learned_addr learned my addr 172.21.15.138:0/1920846699 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.968+0000 7fa350689640 1 -- 172.21.15.138:0/1920846699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a7b56c0 con 0x556d7a7b1000 2024-04-27T05:22:32.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa350689640 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x556d7b669350 tx=0x556d7a8c2d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa34d683640 1 -- 172.21.15.138:0/1920846699 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c9e40 con 0x556d7a7b1000 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa34d683640 1 -- 172.21.15.138:0/1920846699 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7b67c000 con 0x556d7a7b1000 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa34d683640 1 -- 172.21.15.138:0/1920846699 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c94a0 con 0x556d7a7b1000 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d7a7b51e0 con 0x556d7a7b1000 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7a822ea0 con 0x556d7a7b1000 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa354215880 1 -- start start 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa354215880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.969+0000 7fa354215880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7a7f0c80 con 0x556d7b5a9800 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa34d683640 1 -- 172.21.15.138:0/1920846699 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556d7a7db600 con 0x556d7a7b1000 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350e8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350e8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35720/0 (socket says 172.21.15.138:35720) 2024-04-27T05:22:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350e8a640 1 -- 172.21.15.138:0/2950535943 learned_addr learned my addr 172.21.15.138:0/2950535943 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa34d683640 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b718000 0x556d7aa93080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa34d683640 1 -- 172.21.15.138:0/1920846699 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x556d7a7c3a40 con 0x556d7a7b1000 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350689640 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b718000 0x556d7aa93080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350e8a640 1 -- 172.21.15.138:0/2950535943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a822d00 con 0x556d7b5a9800 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350e8a640 1 --2- 172.21.15.138:0/2950535943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x556d7b6bf200 tx=0x556d7a8c3c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418a8a8556ef6345 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa350689640 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b718000 0x556d7aa93080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x556d7b6be4b0 tx=0x556d7a8c34a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa34ae7e640 1 -- 172.21.15.138:0/2950535943 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c9e40 con 0x556d7b5a9800 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa34ae7e640 1 -- 172.21.15.138:0/2950535943 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7b67c480 con 0x556d7b5a9800 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.970+0000 7fa34ae7e640 1 -- 172.21.15.138:0/2950535943 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c94a0 con 0x556d7b5a9800 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- 172.21.15.138:0/2950535943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 msgr2=0x556d7aa92b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 --2- 172.21.15.138:0/2950535943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x556d7b6bf200 tx=0x556d7a8c3c80 comp rx=0 tx=0).stop 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- 172.21.15.138:0/2950535943 shutdown_connections 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 --2- 172.21.15.138:0/2950535943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- 172.21.15.138:0/2950535943 >> 172.21.15.138:0/2950535943 conn(0x556d7a7b1400 msgr2=0x556d7a8bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:32.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- 172.21.15.138:0/2950535943 shutdown_connections 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- 172.21.15.138:0/2950535943 wait complete. 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- start start 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa354215880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7a7d4280 con 0x556d7b5a9800 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa350e8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa350e8a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35728/0 (socket says 172.21.15.138:35728) 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.971+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 learned_addr learned my addr 172.21.15.138:0/3943037483 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a822b60 con 0x556d7b5a9800 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa350e8a640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x556d7b669a70 tx=0x556d7a8c3aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa34967b640 1 -- 172.21.15.138:0/3943037483 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c9e40 con 0x556d7b5a9800 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa34967b640 1 -- 172.21.15.138:0/3943037483 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7b67c600 con 0x556d7b5a9800 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d7a8229c0 con 0x556d7b5a9800 2024-04-27T05:22:32.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa34967b640 1 -- 172.21.15.138:0/3943037483 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7a7c94a0 con 0x556d7b5a9800 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.972+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7a822820 con 0x556d7b5a9800 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa34967b640 1 -- 172.21.15.138:0/3943037483 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556d7a7db8c0 con 0x556d7b5a9800 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa34967b640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7ab46000 0x556d7aa93600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa34967b640 1 -- 172.21.15.138:0/3943037483 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x556d7a7c3c00 con 0x556d7b5a9800 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa34fe88640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7ab46000 0x556d7aa93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x556d7b5a9c00 0x556d7aa93b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46400 con 0x556d7b5a9c00 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa350e8a640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x556d7b5a9c00 0x556d7aa93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa34fe88640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7ab46000 0x556d7aa93600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x556d7a8bcc00 tx=0x556d7b72ad80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.973+0000 7fa350e8a640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x556d7b5a9c00 0x556d7aa93b80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 realm 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46800 con 0x556d7b5a9c00 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46c00 con 0x556d7b5a9c00 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47000 con 0x556d7b5a9c00 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:32.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556d7ab47400 0x556d7aa94100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46400 con 0x556d7ab47400 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.974+0000 7fa350689640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556d7ab47400 0x556d7aa94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa350689640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556d7ab47400 0x556d7aa94100 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa354215880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa354215880 20 searching for the correct realm 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46800 con 0x556d7ab47400 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa350689640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556d7ab46800 0x556d7aa94c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46400 con 0x556d7ab46800 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.975+0000 7fa34fe88640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556d7ab46800 0x556d7aa94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.976+0000 7fa34fe88640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556d7ab46800 0x556d7aa94c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.976+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:32.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.976+0000 7fa34fe88640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556d7ab46400 0x556d7aa95180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.976+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.976+0000 7fa350e8a640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556d7ab46400 0x556d7aa95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.976+0000 7fa350e8a640 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556d7ab46400 0x556d7aa95180 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.977+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46800 2024-04-27T05:22:32.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:32.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:32.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:32.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:32.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:32.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.978+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46800 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.979+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.980+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46800 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.981+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46800 2024-04-27T05:22:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.982+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab47400 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.983+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.983+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46400 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.983+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.983+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7b5a9c00 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.983+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.983+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b5a8800 con 0x556d7ab46800 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46c00 con 0x556d7b5a9c00 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47000 con 0x556d7b5a9c00 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47800 con 0x556d7ab47400 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 zone default found 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 4 Realm: () 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 zonegroup default 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.984+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47c00 con 0x556d7ab46400 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698000 con 0x556d7ab47400 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698400 con 0x556d7ab47400 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 started sync module instance, tier type = 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.985+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698800 con 0x556d7ab46800 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.986+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.986+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93928697669888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698c00 con 0x556d7ab46800 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.986+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b699000 con 0x556d7b5a9c00 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.987+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93928697669888] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.987+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.987+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93928697671040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b699400 con 0x556d7b5a9c00 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.987+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b699800 con 0x556d7ab47400 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.988+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.988+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93928697672192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698000 con 0x556d7ab47400 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.988+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698400 con 0x556d7ab46800 2024-04-27T05:22:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.989+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93928697671040] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.990+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.990+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93928697672192] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.990+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93928697673344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b699800 con 0x556d7ab46800 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.990+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47800 con 0x556d7b5a9c00 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.990+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.991+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93928697674496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab46c00 con 0x556d7b5a9c00 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.991+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47000 con 0x556d7ab46400 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.991+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93928697673344] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.992+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.992+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93928697675648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ab47c00 con 0x556d7ab46400 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.992+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b719800 con 0x556d7ab46400 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.992+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93928697674496] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.992+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93928697675648] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.993+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.993+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93928697676800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698800 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.993+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698c00 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93928697676800] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93928697677952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698400 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=0 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=1 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=2 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=3 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=4 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=5 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.994+0000 7fa354215880 20 add_watcher() i=6 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.995+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93928697677952] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.995+0000 7fa354215880 20 add_watcher() i=7 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.995+0000 7fa354215880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.995+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b699800 con 0x556d7b5a9c00 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.995+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.995+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93928697679104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b699000 con 0x556d7b5a9c00 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93928697679104] v31'204 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa337657640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa354215880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa354215880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa354215880 1 -- start start 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa354215880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.996+0000 7fa354215880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7a7d48c0 con 0x556d7b699c00 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa34fe88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa34fe88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35736/0 (socket says 172.21.15.138:35736) 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa34fe88640 1 -- 172.21.15.138:0/934645059 learned_addr learned my addr 172.21.15.138:0/934645059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa34fe88640 1 -- 172.21.15.138:0/934645059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a822680 con 0x556d7b699c00 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa34fe88640 1 --2- 172.21.15.138:0/934645059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x556d7ab2ac30 tx=0x556d7b72ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f2c2409b96f796 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa336e56640 1 -- 172.21.15.138:0/934645059 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7b69a580 con 0x556d7b699c00 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa336e56640 1 -- 172.21.15.138:0/934645059 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7b67c000 con 0x556d7b699c00 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.997+0000 7fa336e56640 1 -- 172.21.15.138:0/934645059 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7b69a6e0 con 0x556d7b699c00 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 -- 172.21.15.138:0/934645059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 msgr2=0x556d7aa94680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 --2- 172.21.15.138:0/934645059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x556d7ab2ac30 tx=0x556d7b72ade0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 -- 172.21.15.138:0/934645059 shutdown_connections 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 --2- 172.21.15.138:0/934645059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 -- 172.21.15.138:0/934645059 >> 172.21.15.138:0/934645059 conn(0x556d7b699400 msgr2=0x556d7a8bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 -- 172.21.15.138:0/934645059 shutdown_connections 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 -- 172.21.15.138:0/934645059 wait complete. 2024-04-27T05:22:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 Processor -- start 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.998+0000 7fa354215880 1 -- start start 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa354215880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa354215880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7a7d4780 con 0x556d7b699c00 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa34fe88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa34fe88640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35750/0 (socket says 172.21.15.138:35750) 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa34fe88640 1 -- 172.21.15.138:0/1405428372 learned_addr learned my addr 172.21.15.138:0/1405428372 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa34fe88640 1 -- 172.21.15.138:0/1405428372 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7a7b4ea0 con 0x556d7b699c00 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa34fe88640 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x556d7ab2b7a0 tx=0x556d7b72ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa335653640 1 -- 172.21.15.138:0/1405428372 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7b69a840 con 0x556d7b699c00 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa335653640 1 -- 172.21.15.138:0/1405428372 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d7b67c900 con 0x556d7b699c00 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d7a7b56c0 con 0x556d7b699c00 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:32.999+0000 7fa335653640 1 -- 172.21.15.138:0/1405428372 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d7b69a580 con 0x556d7b699c00 2024-04-27T05:22:33.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.000+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7a7b51e0 con 0x556d7b699c00 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.000+0000 7fa335653640 1 -- 172.21.15.138:0/1405428372 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x556d7a7dab00 con 0x556d7b699c00 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.000+0000 7fa335653640 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b698800 0x556d7aa95700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.000+0000 7fa335653640 1 -- 172.21.15.138:0/1405428372 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x556d7a7c3dc0 con 0x556d7b699c00 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.000+0000 7fa354215880 5 note: GC not initialized 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.000+0000 7fa350e8a640 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b698800 0x556d7aa95700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa332e4e640 20 reqs_thread_entry: start 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa354215880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa350e8a640 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b698800 0x556d7aa95700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x556d7b738a20 tx=0x556d7b72b0e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa354215880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa330e4a640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa330628640 20 reqs_thread_entry: start 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32000 con 0x556d7b5a9c00 2024-04-27T05:22:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32400 con 0x556d7ab46800 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.001+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32800 con 0x556d7b5a9c00 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32c00 con 0x556d7ab47400 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33000 con 0x556d7ab47400 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33400 con 0x556d7ab47400 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33800 con 0x556d7b5a9c00 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.002+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33c00 con 0x556d7b5a9c00 2024-04-27T05:22:33.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e000 con 0x556d7ab47400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e400 con 0x556d7ab47400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.003+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e800 con 0x556d7ab46400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.004+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.004+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3ec00 con 0x556d7b5a9c00 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.004+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.004+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3f000 con 0x556d7ab46800 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.004+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.004+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3f400 con 0x556d7ab47400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3f800 con 0x556d7ab46400 2024-04-27T05:22:33.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3fc00 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e800 con 0x556d7ab47400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.005+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e000 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32000 con 0x556d7b5a9c00 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32800 con 0x556d7ab47400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33800 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.006+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33c00 con 0x556d7ab46800 2024-04-27T05:22:33.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32c00 con 0x556d7ab46400 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7ab46400 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33000 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33400 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698c00 con 0x556d7ab47400 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.007+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698400 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a6c0 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7b698000 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7b5a9c00 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3ec00 con 0x556d7ab47400 2024-04-27T05:22:33.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3f800 con 0x556d7ab47400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3fc00 con 0x556d7ab47400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.008+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e000 con 0x556d7ab46400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.009+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7ab46400 2024-04-27T05:22:33.028 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "anonymous", 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "anonymous", 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_buckets", 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 231, 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.029 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.030 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_buckets", 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 322, 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.031 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 5 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.032 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_acls", 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 1026, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.033 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_obj", 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 4, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_bucket", 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 515, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.034 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_acls", 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_obj", 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.035 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.036 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.037 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.038 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.039 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.040 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "anonymous", 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_buckets", 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 231, 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 231, 2024-04-27T05:22:33.041 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 8, 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 8 2024-04-27T05:22:33.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_acls", 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 1026, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_obj", 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 4, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.009+0000 7fa337657640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.009+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93928697679104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e400 con 0x556d7b5a9c00 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.010+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93928697679104] v31'205 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7b5a9c00 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.010+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93928697669888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32000 con 0x556d7ab46800 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.011+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93928697669888] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.011+0000 7fa354215880 20 remove_watcher() i=0 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.011+0000 7fa354215880 2 removed watcher, disabling cache 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.011+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93928697671040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33800 con 0x556d7b5a9c00 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.012+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93928697671040] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7b5a9c00 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.012+0000 7fa354215880 20 remove_watcher() i=1 2024-04-27T05:22:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.012+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93928697672192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac32c00 con 0x556d7ab47400 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.013+0000 7fa350689640 1 -- 172.21.15.138:0/3943037483 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93928697672192] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59ad80 con 0x556d7ab47400 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.013+0000 7fa354215880 20 remove_watcher() i=2 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.013+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93928697673344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac33000 con 0x556d7ab46800 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.014+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93928697673344] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59afc0 con 0x556d7ab46800 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.014+0000 7fa354215880 20 remove_watcher() i=3 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.014+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93928697674496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3f000 con 0x556d7b5a9c00 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.015+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93928697674496] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7b5a9c00 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.015+0000 7fa354215880 20 remove_watcher() i=4 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.015+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93928697675648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3f400 con 0x556d7ab46400 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.016+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93928697675648] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7ab46400 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.016+0000 7fa354215880 20 remove_watcher() i=5 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.016+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93928697676800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3e800 con 0x556d7ab46400 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.017+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93928697676800] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7ab46400 2024-04-27T05:22:33.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.017+0000 7fa354215880 20 remove_watcher() i=6 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.017+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93928697677952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556d7ac3ec00 con 0x556d7ab46400 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa350e8a640 1 -- 172.21.15.138:0/3943037483 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93928697677952] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d7b59a900 con 0x556d7ab46400 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 20 remove_watcher() i=7 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556d7ab46400 msgr2=0x556d7aa95180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x556d7ab46400 0x556d7aa95180 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x556d7b5a9c00 msgr2=0x556d7aa93b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x556d7b5a9c00 0x556d7aa93b80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556d7ab46800 msgr2=0x556d7aa94c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x556d7ab46800 0x556d7aa94c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556d7ab47400 msgr2=0x556d7aa94100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x556d7ab47400 0x556d7aa94100 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7ab46000 msgr2=0x556d7aa93600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7ab46000 0x556d7aa93600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x556d7a8bcc00 tx=0x556d7b72ad80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 msgr2=0x556d7aa92b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa354215880 1 --2- 172.21.15.138:0/3943037483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b5a9800 0x556d7aa92b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x556d7b669a70 tx=0x556d7a8c3aa0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.018+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 reap_dead start 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa34fe88640 1 -- 172.21.15.138:0/3943037483 reap_dead start 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 shutdown_connections 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 >> 172.21.15.138:0/3943037483 conn(0x556d7a7b1400 msgr2=0x556d7a8bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 shutdown_connections 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/3943037483 wait complete. 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b698800 msgr2=0x556d7aa95700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b698800 0x556d7aa95700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x556d7b738a20 tx=0x556d7b72b0e0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 msgr2=0x556d7aa94680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x556d7ab2b7a0 tx=0x556d7b72ac60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 shutdown_connections 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b698800 0x556d7aa95700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 --2- 172.21.15.138:0/1405428372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7b699c00 0x556d7aa94680 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 >> 172.21.15.138:0/1405428372 conn(0x556d7b699400 msgr2=0x556d7a8bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 shutdown_connections 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1405428372 wait complete. 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b718000 msgr2=0x556d7aa93080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b718000 0x556d7aa93080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x556d7b6be4b0 tx=0x556d7a8c34a0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 msgr2=0x556d7aa92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.019+0000 7fa354215880 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x556d7b669350 tx=0x556d7a8c2d20 comp rx=0 tx=0).stop 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.020+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 shutdown_connections 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.020+0000 7fa354215880 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x556d7b718000 0x556d7aa93080 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.020+0000 7fa354215880 1 --2- 172.21.15.138:0/1920846699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7a7b1000 0x556d7aa92580 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.020+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 >> 172.21.15.138:0/1920846699 conn(0x556d7a7b0c00 msgr2=0x556d7a8b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.020+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 shutdown_connections 2024-04-27T05:22:33.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.020+0000 7fa354215880 1 -- 172.21.15.138:0/1920846699 wait complete. 2024-04-27T05:22:33.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.055 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_bucket", 2024-04-27T05:22:33.055 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 515, 2024-04-27T05:22:33.055 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_buckets", 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 322, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_acls", 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.056 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_obj", 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 2103, 2024-04-27T05:22:33.057 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.058 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 35, 2024-04-27T05:22:33.058 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 34 2024-04-27T05:22:33.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.058 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.058 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.058 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-04-27T05:22:33.059 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-27T05:22:33.059 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-27T05:22:33.059 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-27T05:22:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.137 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.135+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.135+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a4c26880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5620b257d400 0x5620b285e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a4c26880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5620b2854600 con 0x5620b257d000 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a4c26880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5620b25bc280 con 0x5620b257d400 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a099a640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5620b257d400 0x5620b285e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a119b640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5620b257d000 0x5620b2684800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40912/0 (socket says 172.21.15.138:40912) 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a119b640 1 -- 172.21.15.138:0/606154379 learned_addr learned my addr 172.21.15.138:0/606154379 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a099a640 1 -- 172.21.15.138:0/606154379 >> v1:172.21.15.114:6789/0 conn(0x5620b257d000 legacy=0x5620b2684800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.136+0000 7f03a099a640 1 -- 172.21.15.138:0/606154379 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b2581380 con 0x5620b257d400 2024-04-27T05:22:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.138+0000 7f03a099a640 1 --2- 172.21.15.138:0/606154379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d400 0x5620b285e580 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5620b2688c00 tx=0x5620b268ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23d51b6668cc646 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.138+0000 7f03a0199640 1 -- 172.21.15.138:0/606154379 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b2595ce0 con 0x5620b257d400 2024-04-27T05:22:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.138+0000 7f03a0199640 1 -- 172.21.15.138:0/606154379 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b2855b00 con 0x5620b257d400 2024-04-27T05:22:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.138+0000 7f03a0199640 1 -- 172.21.15.138:0/606154379 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b2595e40 con 0x5620b257d400 2024-04-27T05:22:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.138+0000 7f03a4c26880 1 -- 172.21.15.138:0/606154379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d400 msgr2=0x5620b285e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.138+0000 7f03a4c26880 1 --2- 172.21.15.138:0/606154379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d400 0x5620b285e580 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5620b2688c00 tx=0x5620b268ede0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 -- 172.21.15.138:0/606154379 shutdown_connections 2024-04-27T05:22:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 --2- 172.21.15.138:0/606154379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d400 0x5620b285e580 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 -- 172.21.15.138:0/606154379 >> 172.21.15.138:0/606154379 conn(0x5620b257cc00 msgr2=0x5620b2684000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 -- 172.21.15.138:0/606154379 shutdown_connections 2024-04-27T05:22:33.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 -- 172.21.15.138:0/606154379 wait complete. 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.139+0000 7f03a4c26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b25bc780 con 0x5620b257d000 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a119b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a119b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35760/0 (socket says 172.21.15.138:35760) 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a119b640 1 -- 172.21.15.138:0/2766328447 learned_addr learned my addr 172.21.15.138:0/2766328447 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a119b640 1 -- 172.21.15.138:0/2766328447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b2580ea0 con 0x5620b257d000 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a119b640 1 --2- 172.21.15.138:0/2766328447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x5620b3438cc0 tx=0x5620b268ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b2f5ed1b9af0f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f039f998640 1 -- 172.21.15.138:0/2766328447 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b25954a0 con 0x5620b257d000 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f039f998640 1 -- 172.21.15.138:0/2766328447 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b2855e00 con 0x5620b257d000 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f039f998640 1 -- 172.21.15.138:0/2766328447 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b2595ce0 con 0x5620b257d000 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a4c26880 1 -- 172.21.15.138:0/2766328447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 msgr2=0x5620b285e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.140+0000 7f03a4c26880 1 --2- 172.21.15.138:0/2766328447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x5620b3438cc0 tx=0x5620b268ecc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 -- 172.21.15.138:0/2766328447 shutdown_connections 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 --2- 172.21.15.138:0/2766328447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 -- 172.21.15.138:0/2766328447 >> 172.21.15.138:0/2766328447 conn(0x5620b257cc00 msgr2=0x5620b2685000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 -- 172.21.15.138:0/2766328447 shutdown_connections 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 -- 172.21.15.138:0/2766328447 wait complete. 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a4c26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b25bc8c0 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a119b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a119b640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35762/0 (socket says 172.21.15.138:35762) 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a119b640 1 -- 172.21.15.138:0/3061145677 learned_addr learned my addr 172.21.15.138:0/3061145677 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.141+0000 7f03a119b640 1 -- 172.21.15.138:0/3061145677 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b25816c0 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a119b640 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5620b3438ea0 tx=0x5620b268ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f039e195640 1 -- 172.21.15.138:0/3061145677 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b2595e40 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f039e195640 1 -- 172.21.15.138:0/3061145677 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b344e000 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5620b25811e0 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f039e195640 1 -- 172.21.15.138:0/3061145677 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b25954a0 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620b25eeea0 con 0x5620b257d000 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a4c26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a4c26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b25bcc80 con 0x5620b3375800 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f039e195640 1 -- 172.21.15.138:0/3061145677 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5620b25a7600 con 0x5620b257d000 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.142+0000 7f03a199c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a199c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35770/0 (socket says 172.21.15.138:35770) 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a199c640 1 -- 172.21.15.138:0/523684448 learned_addr learned my addr 172.21.15.138:0/523684448 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f039e195640 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b34dc000 0x5620b285f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f039e195640 1 -- 172.21.15.138:0/3061145677 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5620b258fa40 con 0x5620b257d000 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a119b640 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b34dc000 0x5620b285f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a199c640 1 -- 172.21.15.138:0/523684448 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b25eed00 con 0x5620b3375800 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a199c640 1 --2- 172.21.15.138:0/523684448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5620b3487440 tx=0x5620b268fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f511eed504d650 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a119b640 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b34dc000 0x5620b285f080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5620b3486480 tx=0x5620b268f4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f039b990640 1 -- 172.21.15.138:0/523684448 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b2595e40 con 0x5620b3375800 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f039b990640 1 -- 172.21.15.138:0/523684448 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b344e300 con 0x5620b3375800 2024-04-27T05:22:33.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f039b990640 1 -- 172.21.15.138:0/523684448 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b25954a0 con 0x5620b3375800 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a4c26880 1 -- 172.21.15.138:0/523684448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 msgr2=0x5620b285eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a4c26880 1 --2- 172.21.15.138:0/523684448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5620b3487440 tx=0x5620b268fc80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.143+0000 7f03a4c26880 1 -- 172.21.15.138:0/523684448 shutdown_connections 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 --2- 172.21.15.138:0/523684448 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 -- 172.21.15.138:0/523684448 >> 172.21.15.138:0/523684448 conn(0x5620b257d400 msgr2=0x5620b2687000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 -- 172.21.15.138:0/523684448 shutdown_connections 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 -- 172.21.15.138:0/523684448 wait complete. 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a4c26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b25a0280 con 0x5620b3375800 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a199c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a199c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35780/0 (socket says 172.21.15.138:35780) 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 learned_addr learned my addr 172.21.15.138:0/556456447 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.144+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b25eeb60 con 0x5620b3375800 2024-04-27T05:22:33.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f03a199c640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5620b34399e0 tx=0x5620b268faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f039a18d640 1 -- 172.21.15.138:0/556456447 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b2595e40 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f039a18d640 1 -- 172.21.15.138:0/556456447 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b344e480 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5620b25ee9c0 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f039a18d640 1 -- 172.21.15.138:0/556456447 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b25954a0 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620b25ee820 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f039a18d640 1 -- 172.21.15.138:0/556456447 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5620b25a78c0 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.145+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f039a18d640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b290a000 0x5620b285f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f039a18d640 1 -- 172.21.15.138:0/556456447 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5620b258fc00 con 0x5620b3375800 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a099a640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b290a000 0x5620b285f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5620b3375c00 0x5620b285fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290a400 con 0x5620b3375c00 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a199c640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5620b3375c00 0x5620b285fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a099a640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b290a000 0x5620b285f600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5620b2688c00 tx=0x5620b34bcd80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a199c640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5620b3375c00 0x5620b285fb80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a4c26880 20 realm 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.146+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290a800 con 0x5620b3375c00 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290ac00 con 0x5620b3375c00 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290b000 con 0x5620b3375c00 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5620b290b400 0x5620b2860100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290a400 con 0x5620b290b400 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a119b640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5620b290b400 0x5620b2860100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.147+0000 7f03a119b640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5620b290b400 0x5620b2860100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a4c26880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a4c26880 20 searching for the correct realm 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290a800 con 0x5620b290b400 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a119b640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5620b290a800 0x5620b2860c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290a400 con 0x5620b290a800 2024-04-27T05:22:33.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a099a640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5620b290a800 0x5620b2860c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.148+0000 7f03a099a640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5620b290a800 0x5620b2860c00 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.149+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.149+0000 7f03a099a640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5620b290a400 0x5620b2861180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.149+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.149+0000 7f03a199c640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5620b290a400 0x5620b2861180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.149+0000 7f03a199c640 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5620b290a400 0x5620b2861180 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.150+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a800 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.151+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.151+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.151+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.151+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.151+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.151+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a800 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.152+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.153+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a800 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.154+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.155+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.155+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.155+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.155+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.155+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.155+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.156+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.156+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a800 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.156+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.156+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290b400 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.156+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.156+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a400 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b3375c00 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3374800 con 0x5620b290a800 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290ac00 con 0x5620b3375c00 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.157+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290b000 con 0x5620b3375c00 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290b800 con 0x5620b290b400 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 zone default found 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 4 Realm: () 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 zonegroup default 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290bc00 con 0x5620b290a400 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360000 con 0x5620b290b400 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.158+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360400 con 0x5620b290b400 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.159+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.159+0000 7f03a4c26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.159+0000 7f03a4c26880 20 started sync module instance, tier type = 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.159+0000 7f03a4c26880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.159+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360800 con 0x5620b290a800 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.160+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.160+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94698434029824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360c00 con 0x5620b290a800 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.160+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3361000 con 0x5620b3375c00 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.161+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94698434029824] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.161+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.161+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94698434030976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3361400 con 0x5620b3375c00 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.161+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3361800 con 0x5620b290b400 2024-04-27T05:22:33.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.162+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94698434030976] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.162+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.162+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94698434032128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360000 con 0x5620b290b400 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.162+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360400 con 0x5620b290a800 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.163+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94698434032128] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.163+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.163+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94698434033280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3361800 con 0x5620b290a800 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.163+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290b800 con 0x5620b3375c00 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.164+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.164+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94698434034432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290ac00 con 0x5620b3375c00 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.164+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290b000 con 0x5620b290a400 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.166+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94698434033280] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.166+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.166+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94698434035584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b290bc00 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.166+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b34dd800 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.166+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94698434034432] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.167+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.167+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94698434036736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360800 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.167+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360c00 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.167+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94698434035584] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94698434036736] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94698434037888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3360400 con 0x5620b290a400 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=0 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=1 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=2 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=3 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=4 2024-04-27T05:22:33.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=5 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.168+0000 7f03a4c26880 20 add_watcher() i=6 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.169+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94698434037888] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.169+0000 7f03a4c26880 20 add_watcher() i=7 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.169+0000 7f03a4c26880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.169+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3361800 con 0x5620b3375c00 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.169+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.169+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94698434039040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b3361000 con 0x5620b3375c00 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94698434039040] v31'206 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f0388169640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a4c26880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a4c26880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a4c26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.170+0000 7f03a4c26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b25a08c0 con 0x5620b3361c00 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f03a099a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f03a099a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35786/0 (socket says 172.21.15.138:35786) 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f03a099a640 1 -- 172.21.15.138:0/3556495921 learned_addr learned my addr 172.21.15.138:0/3556495921 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f03a099a640 1 -- 172.21.15.138:0/3556495921 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b25ee680 con 0x5620b3361c00 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f03a099a640 1 --2- 172.21.15.138:0/3556495921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5620b34f2780 tx=0x5620b34bcde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee6a01692988a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f0387968640 1 -- 172.21.15.138:0/3556495921 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b3362580 con 0x5620b3361c00 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f0387968640 1 -- 172.21.15.138:0/3556495921 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b344e000 con 0x5620b3361c00 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.171+0000 7f0387968640 1 -- 172.21.15.138:0/3556495921 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b33626e0 con 0x5620b3361c00 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- 172.21.15.138:0/3556495921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 msgr2=0x5620b2860680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 --2- 172.21.15.138:0/3556495921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5620b34f2780 tx=0x5620b34bcde0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- 172.21.15.138:0/3556495921 shutdown_connections 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 --2- 172.21.15.138:0/3556495921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- 172.21.15.138:0/3556495921 >> 172.21.15.138:0/3556495921 conn(0x5620b3361400 msgr2=0x5620b2687800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- 172.21.15.138:0/3556495921 shutdown_connections 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- 172.21.15.138:0/3556495921 wait complete. 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 Processor -- start 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- start start 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.172+0000 7f03a4c26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b25a0780 con 0x5620b3361c00 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a099a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a099a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35790/0 (socket says 172.21.15.138:35790) 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a099a640 1 -- 172.21.15.138:0/440798024 learned_addr learned my addr 172.21.15.138:0/440798024 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a099a640 1 -- 172.21.15.138:0/440798024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b25ee4e0 con 0x5620b3361c00 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a099a640 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x5620b28e3740 tx=0x5620b34bcc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f0386165640 1 -- 172.21.15.138:0/440798024 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b3362840 con 0x5620b3361c00 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f0386165640 1 -- 172.21.15.138:0/440798024 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5620b344e780 con 0x5620b3361c00 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5620b2580ea0 con 0x5620b3361c00 2024-04-27T05:22:33.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f0386165640 1 -- 172.21.15.138:0/440798024 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5620b3362580 con 0x5620b3361c00 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.173+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620b25816c0 con 0x5620b3361c00 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.174+0000 7f0386165640 1 -- 172.21.15.138:0/440798024 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5620b25a6b00 con 0x5620b3361c00 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.174+0000 7f0386165640 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b3360800 0x5620b2861700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.174+0000 7f0386165640 1 -- 172.21.15.138:0/440798024 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5620b258fdc0 con 0x5620b3361c00 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.174+0000 7f03a4c26880 5 note: GC not initialized 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.174+0000 7f03a199c640 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b3360800 0x5620b2861700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f0383960640 20 reqs_thread_entry: start 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a4c26880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a199c640 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b3360800 0x5620b2861700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5620b34f2bd0 tx=0x5620b34bd0e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a4c26880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f038195c640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f038113a640 20 reqs_thread_entry: start 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a4c26880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4000 con 0x5620b3375c00 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a4c26880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.175+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4400 con 0x5620b290a400 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.176+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.191 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_buckets", 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 322, 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 5 2024-04-27T05:22:33.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_acls", 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 1026, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_obj", 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 4, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_bucket", 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 515, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_acls", 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_obj", 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.197 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.198 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.177+0000 7f0388169640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.177+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94698434039040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4800 con 0x5620b3375c00 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.178+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94698434039040] v31'207 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.178+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94698434029824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4c00 con 0x5620b290a800 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.179+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94698434029824] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.179+0000 7f03a4c26880 20 remove_watcher() i=0 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.179+0000 7f03a4c26880 2 removed watcher, disabling cache 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.179+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94698434030976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f5000 con 0x5620b3375c00 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.180+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94698434030976] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.180+0000 7f03a4c26880 20 remove_watcher() i=1 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.180+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94698434032128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f5400 con 0x5620b290b400 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.181+0000 7f03a119b640 1 -- 172.21.15.138:0/556456447 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94698434032128] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916fc0 con 0x5620b290b400 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.181+0000 7f03a4c26880 20 remove_watcher() i=2 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.181+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94698434033280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f5800 con 0x5620b290a800 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.182+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94698434033280] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2917200 con 0x5620b290a800 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.182+0000 7f03a4c26880 20 remove_watcher() i=3 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.182+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94698434034432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f5c00 con 0x5620b3375c00 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.183+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94698434034432] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b3375c00 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.183+0000 7f03a4c26880 20 remove_watcher() i=4 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.183+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94698434035584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4000 con 0x5620b290a400 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.184+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94698434035584] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.184+0000 7f03a4c26880 20 remove_watcher() i=5 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.184+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94698434036736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4400 con 0x5620b290a400 2024-04-27T05:22:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.185+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94698434036736] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.185+0000 7f03a4c26880 20 remove_watcher() i=6 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.185+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94698434037888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5620b29f4800 con 0x5620b290a400 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a199c640 1 -- 172.21.15.138:0/556456447 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94698434037888] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5620b2916900 con 0x5620b290a400 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 20 remove_watcher() i=7 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5620b290a400 msgr2=0x5620b2861180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5620b290a400 0x5620b2861180 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5620b3375c00 msgr2=0x5620b285fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5620b3375c00 0x5620b285fb80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5620b290a800 msgr2=0x5620b2860c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5620b290a800 0x5620b2860c00 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5620b290b400 msgr2=0x5620b2860100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5620b290b400 0x5620b2860100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b290a000 msgr2=0x5620b285f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b290a000 0x5620b285f600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5620b2688c00 tx=0x5620b34bcd80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 msgr2=0x5620b285eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/556456447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3375800 0x5620b285eb00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5620b34399e0 tx=0x5620b268faa0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 reap_dead start 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a099a640 1 -- 172.21.15.138:0/556456447 reap_dead start 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 shutdown_connections 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 >> 172.21.15.138:0/556456447 conn(0x5620b257d400 msgr2=0x5620b2687000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 shutdown_connections 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/556456447 wait complete. 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b3360800 msgr2=0x5620b2861700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b3360800 0x5620b2861700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5620b34f2bd0 tx=0x5620b34bd0e0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 msgr2=0x5620b2860680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x5620b28e3740 tx=0x5620b34bcc60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 shutdown_connections 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b3360800 0x5620b2861700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 --2- 172.21.15.138:0/440798024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b3361c00 0x5620b2860680 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 >> 172.21.15.138:0/440798024 conn(0x5620b3361400 msgr2=0x5620b2687800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.186+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 shutdown_connections 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/440798024 wait complete. 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b34dc000 msgr2=0x5620b285f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b34dc000 0x5620b285f080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5620b3486480 tx=0x5620b268f4a0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 msgr2=0x5620b285e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5620b3438ea0 tx=0x5620b268ec60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 shutdown_connections 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5620b34dc000 0x5620b285f080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 --2- 172.21.15.138:0/3061145677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5620b257d000 0x5620b285e580 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 >> 172.21.15.138:0/3061145677 conn(0x5620b257cc00 msgr2=0x5620b2685000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 shutdown_connections 2024-04-27T05:22:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.187+0000 7f03a4c26880 1 -- 172.21.15.138:0/3061145677 wait complete. 2024-04-27T05:22:33.209 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:33.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.209 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.209 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.209 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.211 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:33.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 8, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 8 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.213 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_acls", 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 1026, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.214 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "get_obj", 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 4, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_bucket", 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 515, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.215 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "list_buckets", 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 322, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_acls", 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.216 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 2 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_obj", 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 2103, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 35, 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 34 2024-04-27T05:22:33.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.218 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.218 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.218 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-04-27T05:22:33.218 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-27T05:22:33.218 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-27T05:22:33.218 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-04-27T05:22:33.295 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.295 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b9462880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55e996fa7400 0x55e997288580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b9462880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e99727e600 con 0x55e996fa7000 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b9462880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e996fe6280 con 0x55e996fa7400 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b7406640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55e996fa7400 0x55e997288580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b7c07640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55e996fa7000 0x55e9970ae800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40920/0 (socket says 172.21.15.138:40920) 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.292+0000 7fe5b7c07640 1 -- 172.21.15.138:0/4175125627 learned_addr learned my addr 172.21.15.138:0/4175125627 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:33.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.293+0000 7fe5b7406640 1 -- 172.21.15.138:0/4175125627 >> v1:172.21.15.114:6789/0 conn(0x55e996fa7000 legacy=0x55e9970ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.293+0000 7fe5b7406640 1 -- 172.21.15.138:0/4175125627 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e996fab380 con 0x55e996fa7400 2024-04-27T05:22:33.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.293+0000 7fe5b6c05640 1 -- 172.21.15.138:0/4175125627 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352265193 0 0) 0x55e996faa9c0 con 0x55e996fa7000 2024-04-27T05:22:33.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.294+0000 7fe5b7406640 1 --2- 172.21.15.138:0/4175125627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7400 0x55e997288580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55e9970b2c00 tx=0x55e9970b8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f174eb06a2c456 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.294+0000 7fe5b6c05640 1 -- 172.21.15.138:0/4175125627 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbfce0 con 0x55e996fa7400 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.294+0000 7fe5b6c05640 1 -- 172.21.15.138:0/4175125627 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e99727fb00 con 0x55e996fa7400 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.294+0000 7fe5b6c05640 1 -- 172.21.15.138:0/4175125627 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbfe40 con 0x55e996fa7400 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 -- 172.21.15.138:0/4175125627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7400 msgr2=0x55e997288580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 --2- 172.21.15.138:0/4175125627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7400 0x55e997288580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x55e9970b2c00 tx=0x55e9970b8de0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 -- 172.21.15.138:0/4175125627 shutdown_connections 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 --2- 172.21.15.138:0/4175125627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7400 0x55e997288580 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 -- 172.21.15.138:0/4175125627 >> 172.21.15.138:0/4175125627 conn(0x55e996fa6c00 msgr2=0x55e9970ae000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 -- 172.21.15.138:0/4175125627 shutdown_connections 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 -- 172.21.15.138:0/4175125627 wait complete. 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.295+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b9462880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b9462880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996fe6780 con 0x55e996fa7000 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b7c07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b7c07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35818/0 (socket says 172.21.15.138:35818) 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b7c07640 1 -- 172.21.15.138:0/2802117981 learned_addr learned my addr 172.21.15.138:0/2802117981 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b7c07640 1 -- 172.21.15.138:0/2802117981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e996faaea0 con 0x55e996fa7000 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/2802117981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55e997e62c90 tx=0x55e9970b8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f434edd32d03849f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b6404640 1 -- 172.21.15.138:0/2802117981 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbf4a0 con 0x55e996fa7000 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b6404640 1 -- 172.21.15.138:0/2802117981 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e99727fe00 con 0x55e996fa7000 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.296+0000 7fe5b6404640 1 -- 172.21.15.138:0/2802117981 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbfce0 con 0x55e996fa7000 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- 172.21.15.138:0/2802117981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 msgr2=0x55e997288580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 --2- 172.21.15.138:0/2802117981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x55e997e62c90 tx=0x55e9970b8cc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- 172.21.15.138:0/2802117981 shutdown_connections 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 --2- 172.21.15.138:0/2802117981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- 172.21.15.138:0/2802117981 >> 172.21.15.138:0/2802117981 conn(0x55e996fa6c00 msgr2=0x55e9970af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- 172.21.15.138:0/2802117981 shutdown_connections 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- 172.21.15.138:0/2802117981 wait complete. 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b9462880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996fe68c0 con 0x55e996fa7000 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b7c07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b7c07640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35830/0 (socket says 172.21.15.138:35830) 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.297+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1230000769 learned_addr learned my addr 172.21.15.138:0/1230000769 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1230000769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e996fab6c0 con 0x55e996fa7000 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55e997e63350 tx=0x55e9970b8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b4c01640 1 -- 172.21.15.138:0/1230000769 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbfe40 con 0x55e996fa7000 2024-04-27T05:22:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b4c01640 1 -- 172.21.15.138:0/1230000769 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e997e78000 con 0x55e996fa7000 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e996fab1e0 con 0x55e996fa7000 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b4c01640 1 -- 172.21.15.138:0/1230000769 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbf4a0 con 0x55e996fa7000 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e997018ea0 con 0x55e996fa7000 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.298+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b9462880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b9462880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996fe6c80 con 0x55e997d9f800 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b4c01640 1 -- 172.21.15.138:0/1230000769 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e996fd1600 con 0x55e996fa7000 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b8408640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b8408640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35842/0 (socket says 172.21.15.138:35842) 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b8408640 1 -- 172.21.15.138:0/1250602194 learned_addr learned my addr 172.21.15.138:0/1250602194 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b4c01640 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997f06000 0x55e997289080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b4c01640 1 -- 172.21.15.138:0/1230000769 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55e996fb9a40 con 0x55e996fa7000 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997f06000 0x55e997289080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b8408640 1 -- 172.21.15.138:0/1250602194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e997018d00 con 0x55e997d9f800 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b8408640 1 --2- 172.21.15.138:0/1250602194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55e997eb3440 tx=0x55e9970b9c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd07a72e0eeadb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997f06000 0x55e997289080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e997eb2450 tx=0x55e9970b94a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b23fc640 1 -- 172.21.15.138:0/1250602194 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbfe40 con 0x55e997d9f800 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b23fc640 1 -- 172.21.15.138:0/1250602194 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e997e78300 con 0x55e997d9f800 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.299+0000 7fe5b23fc640 1 -- 172.21.15.138:0/1250602194 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbf4a0 con 0x55e997d9f800 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- 172.21.15.138:0/1250602194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 msgr2=0x55e997288b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1250602194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55e997eb3440 tx=0x55e9970b9c80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- 172.21.15.138:0/1250602194 shutdown_connections 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1250602194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- 172.21.15.138:0/1250602194 >> 172.21.15.138:0/1250602194 conn(0x55e996fa7400 msgr2=0x55e9970b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- 172.21.15.138:0/1250602194 shutdown_connections 2024-04-27T05:22:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- 172.21.15.138:0/1250602194 wait complete. 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b9462880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996fca280 con 0x55e997d9f800 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b8408640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b8408640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35858/0 (socket says 172.21.15.138:35858) 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.300+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 learned_addr learned my addr 172.21.15.138:0/1005815820 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e997018b60 con 0x55e997d9f800 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b8408640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55e997e63a70 tx=0x55e9970b9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b0bf9640 1 -- 172.21.15.138:0/1005815820 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbfe40 con 0x55e997d9f800 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b0bf9640 1 -- 172.21.15.138:0/1005815820 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e997e78480 con 0x55e997d9f800 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e9970189c0 con 0x55e997d9f800 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b0bf9640 1 -- 172.21.15.138:0/1005815820 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e996fbf4a0 con 0x55e997d9f800 2024-04-27T05:22:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.301+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e997018820 con 0x55e997d9f800 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b0bf9640 1 -- 172.21.15.138:0/1005815820 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e996fd18c0 con 0x55e997d9f800 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b0bf9640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997330000 0x55e997289600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b0bf9640 1 -- 172.21.15.138:0/1005815820 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55e996fb9c00 con 0x55e997d9f800 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b7406640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997330000 0x55e997289600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55e997d9fc00 0x55e997289b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330400 con 0x55e997d9fc00 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b8408640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55e997d9fc00 0x55e997289b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b7406640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997330000 0x55e997289600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55e9970b2c00 tx=0x55e997eeed80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.302+0000 7fe5b8408640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55e997d9fc00 0x55e997289b80 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 realm 2024-04-27T05:22:33.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330800 con 0x55e997d9fc00 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330c00 con 0x55e997d9fc00 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331000 con 0x55e997d9fc00 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55e997331400 0x55e99728a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330400 con 0x55e997331400 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.303+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55e997331400 0x55e99728a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55e997331400 0x55e99728a100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b9462880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b9462880 20 searching for the correct realm 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330800 con 0x55e997331400 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7c07640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55e997330800 0x55e99728ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330400 con 0x55e997330800 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7406640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55e997330800 0x55e99728ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.304+0000 7fe5b7406640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55e997330800 0x55e99728ac00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.305+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.305+0000 7fe5b7406640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55e997330400 0x55e99728b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.305+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.305+0000 7fe5b8408640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55e997330400 0x55e99728b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.305+0000 7fe5b8408640 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55e997330400 0x55e99728b180 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.306+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330800 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.307+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330800 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.308+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.309+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330800 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.310+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330800 2024-04-27T05:22:33.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997331400 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.311+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330400 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997d9fc00 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d9e800 con 0x55e997330800 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b9462880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b9462880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b9462880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b9462880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.312+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330c00 con 0x55e997d9fc00 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331000 con 0x55e997d9fc00 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331800 con 0x55e997331400 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 zone default found 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 4 Realm: () 2024-04-27T05:22:33.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 zonegroup default 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331c00 con 0x55e997330400 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.313+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86000 con 0x55e997331400 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86400 con 0x55e997331400 2024-04-27T05:22:33.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 20 started sync module instance, tier type = 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.314+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86800 con 0x55e997330800 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.315+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.315+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94461751724288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86c00 con 0x55e997330800 2024-04-27T05:22:33.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.315+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d87000 con 0x55e997d9fc00 2024-04-27T05:22:33.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.316+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94461751724288] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.316+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.316+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94461751725440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d87400 con 0x55e997d9fc00 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.316+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d87800 con 0x55e997331400 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.317+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94461751725440] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.318+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.318+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94461751726592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86000 con 0x55e997331400 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.318+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86400 con 0x55e997330800 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.319+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.319+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94461751726592] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.319+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94461751727744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d87800 con 0x55e997330800 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.319+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331800 con 0x55e997d9fc00 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.320+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.320+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94461751728896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997330c00 con 0x55e997d9fc00 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.320+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331000 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.320+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94461751727744] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.321+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94461751728896] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.321+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.321+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94461751730048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997331c00 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.321+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997f07800 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.322+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.322+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94461751731200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86800 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.322+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86c00 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.322+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94461751730048] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94461751731200] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94461751732352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d86400 con 0x55e997330400 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=0 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=1 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=2 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=3 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=4 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=5 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.323+0000 7fe5b9462880 20 add_watcher() i=6 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.324+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94461751732352] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.324+0000 7fe5b9462880 20 add_watcher() i=7 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.324+0000 7fe5b9462880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.324+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d87800 con 0x55e997d9fc00 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.324+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.324+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94461751733504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e997d87000 con 0x55e997d9fc00 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.325+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94461751733504] v31'208 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe59ebd5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b9462880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b9462880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b9462880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b9462880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996fca8c0 con 0x55e997d87c00 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b7406640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b7406640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35870/0 (socket says 172.21.15.138:35870) 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b7406640 1 -- 172.21.15.138:0/346790772 learned_addr learned my addr 172.21.15.138:0/346790772 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.326+0000 7fe5b7406640 1 -- 172.21.15.138:0/346790772 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e997018680 con 0x55e997d87c00 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe5b7406640 1 --2- 172.21.15.138:0/346790772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55e997312c30 tx=0x55e997eeede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4826953e19872e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe59e3d4640 1 -- 172.21.15.138:0/346790772 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e997d88580 con 0x55e997d87c00 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe59e3d4640 1 -- 172.21.15.138:0/346790772 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e997e78000 con 0x55e997d87c00 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe59e3d4640 1 -- 172.21.15.138:0/346790772 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e997d886e0 con 0x55e997d87c00 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe5b9462880 1 -- 172.21.15.138:0/346790772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 msgr2=0x55e99728a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe5b9462880 1 --2- 172.21.15.138:0/346790772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x55e997312c30 tx=0x55e997eeede0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe5b9462880 1 -- 172.21.15.138:0/346790772 shutdown_connections 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe5b9462880 1 --2- 172.21.15.138:0/346790772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.327+0000 7fe5b9462880 1 -- 172.21.15.138:0/346790772 >> 172.21.15.138:0/346790772 conn(0x55e997d87400 msgr2=0x55e9970b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b9462880 1 -- 172.21.15.138:0/346790772 shutdown_connections 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b9462880 1 -- 172.21.15.138:0/346790772 wait complete. 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b9462880 1 Processor -- start 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b9462880 1 -- start start 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b9462880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b9462880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e996fca780 con 0x55e997d87c00 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b7406640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b7406640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35872/0 (socket says 172.21.15.138:35872) 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b7406640 1 -- 172.21.15.138:0/777109959 learned_addr learned my addr 172.21.15.138:0/777109959 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.328+0000 7fe5b7406640 1 -- 172.21.15.138:0/777109959 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e996faaea0 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe5b7406640 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55e9973137a0 tx=0x55e997eeec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe59cbd1640 1 -- 172.21.15.138:0/777109959 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e997d88840 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe59cbd1640 1 -- 172.21.15.138:0/777109959 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e997e78780 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e996fab6c0 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe59cbd1640 1 -- 172.21.15.138:0/777109959 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e997d88580 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e996fab1e0 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.329+0000 7fe59cbd1640 1 -- 172.21.15.138:0/777109959 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55e996fd0b00 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe59cbd1640 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997d86800 0x55e99728b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe59cbd1640 1 -- 172.21.15.138:0/777109959 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55e996fb9dc0 con 0x55e997d87c00 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b8408640 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997d86800 0x55e99728b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b9462880 5 note: GC not initialized 2024-04-27T05:22:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe59a3cc640 20 reqs_thread_entry: start 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b9462880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b8408640 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997d86800 0x55e99728b700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55e997f24a20 tx=0x55e997eef0e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b9462880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5983c8640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe597ba6640 20 reqs_thread_entry: start 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b9462880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.330+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741e000 con 0x55e997d9fc00 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.331+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.331+0000 7fe5b9462880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.331+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741e400 con 0x55e997330400 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.331+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.332+0000 7fe59ebd5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.332+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94461751733504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741e800 con 0x55e997d9fc00 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.333+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94461751733504] v31'209 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.333+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94461751724288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741ec00 con 0x55e997330800 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.334+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94461751724288] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.334+0000 7fe5b9462880 20 remove_watcher() i=0 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.334+0000 7fe5b9462880 2 removed watcher, disabling cache 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.334+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94461751725440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741f000 con 0x55e997d9fc00 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.335+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94461751725440] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.335+0000 7fe5b9462880 20 remove_watcher() i=1 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.335+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94461751726592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741f400 con 0x55e997331400 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.336+0000 7fe5b7c07640 1 -- 172.21.15.138:0/1005815820 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94461751726592] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348fc0 con 0x55e997331400 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.336+0000 7fe5b9462880 20 remove_watcher() i=2 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.336+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94461751727744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741f800 con 0x55e997330800 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.337+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94461751727744] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997349200 con 0x55e997330800 2024-04-27T05:22:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.337+0000 7fe5b9462880 20 remove_watcher() i=3 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.337+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94461751728896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741fc00 con 0x55e997d9fc00 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.338+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94461751728896] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997d9fc00 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.338+0000 7fe5b9462880 20 remove_watcher() i=4 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.338+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94461751730048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741e000 con 0x55e997330400 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.338+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94461751730048] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.339+0000 7fe5b9462880 20 remove_watcher() i=5 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.339+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94461751731200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741e400 con 0x55e997330400 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.339+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94461751731200] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.339+0000 7fe5b9462880 20 remove_watcher() i=6 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.339+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94461751732352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e99741e800 con 0x55e997330400 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b8408640 1 -- 172.21.15.138:0/1005815820 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94461751732352] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e997348900 con 0x55e997330400 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 20 remove_watcher() i=7 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55e997330400 msgr2=0x55e99728b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55e997330400 0x55e99728b180 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55e997d9fc00 msgr2=0x55e997289b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55e997d9fc00 0x55e997289b80 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55e997330800 msgr2=0x55e99728ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55e997330800 0x55e99728ac00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55e997331400 msgr2=0x55e99728a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55e997331400 0x55e99728a100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997330000 msgr2=0x55e997289600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997330000 0x55e997289600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55e9970b2c00 tx=0x55e997eeed80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 msgr2=0x55e997288b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1005815820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d9f800 0x55e997288b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55e997e63a70 tx=0x55e9970b9aa0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.340+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 reap_dead start 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b7406640 1 -- 172.21.15.138:0/1005815820 reap_dead start 2024-04-27T05:22:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 shutdown_connections 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 >> 172.21.15.138:0/1005815820 conn(0x55e996fa7400 msgr2=0x55e9970b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 shutdown_connections 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1005815820 wait complete. 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997d86800 msgr2=0x55e99728b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997d86800 0x55e99728b700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55e997f24a20 tx=0x55e997eef0e0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 msgr2=0x55e99728a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x55e9973137a0 tx=0x55e997eeec60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 shutdown_connections 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997d86800 0x55e99728b700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/777109959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e997d87c00 0x55e99728a680 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 >> 172.21.15.138:0/777109959 conn(0x55e997d87400 msgr2=0x55e9970b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 shutdown_connections 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/777109959 wait complete. 2024-04-27T05:22:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997f06000 msgr2=0x55e997289080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997f06000 0x55e997289080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e997eb2450 tx=0x55e9970b94a0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 msgr2=0x55e997288580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55e997e63350 tx=0x55e9970b8c60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 shutdown_connections 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55e997f06000 0x55e997289080 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 --2- 172.21.15.138:0/1230000769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e996fa7000 0x55e997288580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.341+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 >> 172.21.15.138:0/1230000769 conn(0x55e996fa6c00 msgr2=0x55e9970af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.342+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 shutdown_connections 2024-04-27T05:22:33.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.342+0000 7fe5b9462880 1 -- 172.21.15.138:0/1230000769 wait complete. 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.360 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.361 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 5 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.362 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.364 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.365 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "create_bucket", 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 8, 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 8 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 8, 2024-04-27T05:22:33.366 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 8 2024-04-27T05:22:33.367 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.367 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.367 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.367 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.367 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}}]} 2024-04-27T05:22:33.367 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-27T05:22:33.367 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-27T05:22:33.367 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-04-27T05:22:33.411 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.411 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c61923880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c61923880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bce5730600 con 0x55bce5459400 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c61923880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bce5498280 con 0x55bce5459000 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c5e269640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c5da68640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55bce5459400 0x55bce5560800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40930/0 (socket says 172.21.15.138:40930) 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.410+0000 7f7c5da68640 1 -- 172.21.15.138:0/933975690 learned_addr learned my addr 172.21.15.138:0/933975690 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.411+0000 7f7c5e269640 1 -- 172.21.15.138:0/933975690 >> v1:172.21.15.114:6789/0 conn(0x55bce5459400 legacy=0x55bce5560800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.411+0000 7f7c5e269640 1 -- 172.21.15.138:0/933975690 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce545d380 con 0x55bce5459000 2024-04-27T05:22:33.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.411+0000 7f7c5d267640 1 -- 172.21.15.138:0/933975690 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263239360 0 0) 0x55bce545c9c0 con 0x55bce5459400 2024-04-27T05:22:33.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.412+0000 7f7c5e269640 1 --2- 172.21.15.138:0/933975690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55bce5564d50 tx=0x55bce556ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54def69520a2e3b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.412+0000 7f7c5d267640 1 -- 172.21.15.138:0/933975690 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5471ce0 con 0x55bce5459000 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.412+0000 7f7c5d267640 1 -- 172.21.15.138:0/933975690 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce5731b00 con 0x55bce5459000 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c5d267640 1 -- 172.21.15.138:0/933975690 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5471e40 con 0x55bce5459000 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 -- 172.21.15.138:0/933975690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 msgr2=0x55bce573a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 --2- 172.21.15.138:0/933975690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55bce5564d50 tx=0x55bce556ade0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 -- 172.21.15.138:0/933975690 shutdown_connections 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 --2- 172.21.15.138:0/933975690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 -- 172.21.15.138:0/933975690 >> 172.21.15.138:0/933975690 conn(0x55bce5458c00 msgr2=0x55bce5560000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 -- 172.21.15.138:0/933975690 shutdown_connections 2024-04-27T05:22:33.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.413+0000 7f7c61923880 1 -- 172.21.15.138:0/933975690 wait complete. 2024-04-27T05:22:33.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c61923880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c61923880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce5498780 con 0x55bce5459000 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c5e269640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c5e269640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35894/0 (socket says 172.21.15.138:35894) 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.414+0000 7f7c5e269640 1 -- 172.21.15.138:0/1355376883 learned_addr learned my addr 172.21.15.138:0/1355376883 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.415+0000 7f7c5e269640 1 -- 172.21.15.138:0/1355376883 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce545cea0 con 0x55bce5459000 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.415+0000 7f7c5e269640 1 --2- 172.21.15.138:0/1355376883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55bce625f590 tx=0x55bce556acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d494a3fe82e60d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.415+0000 7f7c5ca66640 1 -- 172.21.15.138:0/1355376883 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce54714a0 con 0x55bce5459000 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.415+0000 7f7c5ca66640 1 -- 172.21.15.138:0/1355376883 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce5731e00 con 0x55bce5459000 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.415+0000 7f7c5ca66640 1 -- 172.21.15.138:0/1355376883 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5471ce0 con 0x55bce5459000 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- 172.21.15.138:0/1355376883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 msgr2=0x55bce573a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 --2- 172.21.15.138:0/1355376883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55bce625f590 tx=0x55bce556acc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- 172.21.15.138:0/1355376883 shutdown_connections 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 --2- 172.21.15.138:0/1355376883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- 172.21.15.138:0/1355376883 >> 172.21.15.138:0/1355376883 conn(0x55bce5458c00 msgr2=0x55bce5561000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- 172.21.15.138:0/1355376883 shutdown_connections 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- 172.21.15.138:0/1355376883 wait complete. 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c61923880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce54988c0 con 0x55bce5459000 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c5e269640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c5e269640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35900/0 (socket says 172.21.15.138:35900) 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.416+0000 7f7c5e269640 1 -- 172.21.15.138:0/3905975617 learned_addr learned my addr 172.21.15.138:0/3905975617 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c5e269640 1 -- 172.21.15.138:0/3905975617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce545d6c0 con 0x55bce5459000 2024-04-27T05:22:33.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c5e269640 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55bce6314e10 tx=0x55bce556ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c5b263640 1 -- 172.21.15.138:0/3905975617 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5471e40 con 0x55bce5459000 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c5b263640 1 -- 172.21.15.138:0/3905975617 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce6316000 con 0x55bce5459000 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bce545d1e0 con 0x55bce5459000 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c5b263640 1 -- 172.21.15.138:0/3905975617 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce54714a0 con 0x55bce5459000 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bce54caea0 con 0x55bce5459000 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.417+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c61923880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c61923880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce5498c80 con 0x55bce6250c00 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5b263640 1 -- 172.21.15.138:0/3905975617 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bce5482dc0 con 0x55bce5459000 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5ea6a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5b263640 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce6251c00 0x55bce573b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5ea6a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35904/0 (socket says 172.21.15.138:35904) 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2748760706 learned_addr learned my addr 172.21.15.138:0/2748760706 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5b263640 1 -- 172.21.15.138:0/3905975617 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bce546ba40 con 0x55bce5459000 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5e269640 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce6251c00 0x55bce573b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2748760706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce54cad00 con 0x55bce6250c00 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5e269640 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce6251c00 0x55bce573b080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55bce63582d0 tx=0x55bce556bc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/2748760706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55bce6314d20 tx=0x55bce556bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecb2c933e8d8019 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.418+0000 7f7c58a5e640 1 -- 172.21.15.138:0/2748760706 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5471e40 con 0x55bce6250c00 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c58a5e640 1 -- 172.21.15.138:0/2748760706 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce6316600 con 0x55bce6250c00 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c58a5e640 1 -- 172.21.15.138:0/2748760706 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce54714a0 con 0x55bce6250c00 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- 172.21.15.138:0/2748760706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 msgr2=0x55bce573ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 --2- 172.21.15.138:0/2748760706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55bce6314d20 tx=0x55bce556bf80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- 172.21.15.138:0/2748760706 shutdown_connections 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 --2- 172.21.15.138:0/2748760706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- 172.21.15.138:0/2748760706 >> 172.21.15.138:0/2748760706 conn(0x55bce5459400 msgr2=0x55bce5563000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- 172.21.15.138:0/2748760706 shutdown_connections 2024-04-27T05:22:33.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- 172.21.15.138:0/2748760706 wait complete. 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.419+0000 7f7c61923880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce547c280 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5ea6a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5ea6a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35908/0 (socket says 172.21.15.138:35908) 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 learned_addr learned my addr 172.21.15.138:0/2969897650 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce54cab60 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55bce6315410 tx=0x55bce556b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5725b640 1 -- 172.21.15.138:0/2969897650 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5471e40 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5725b640 1 -- 172.21.15.138:0/2969897650 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce6316780 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bce54ca9c0 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c5725b640 1 -- 172.21.15.138:0/2969897650 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce54714a0 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.420+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bce54ca820 con 0x55bce6250c00 2024-04-27T05:22:33.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5725b640 1 -- 172.21.15.138:0/2969897650 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bce5482b00 con 0x55bce6250c00 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5725b640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce57d6000 0x55bce573b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5725b640 1 -- 172.21.15.138:0/2969897650 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bce546bc00 con 0x55bce6250c00 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5da68640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce57d6000 0x55bce573b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bce6251000 0x55bce573bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce6251400 con 0x55bce6251000 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bce6251000 0x55bce573bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5da68640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce57d6000 0x55bce573b600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55bce5565410 tx=0x55bce63c8f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.421+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bce6251000 0x55bce573bb80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 realm 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce6251800 con 0x55bce6251000 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7000 con 0x55bce6251000 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7400 con 0x55bce6251000 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bce57d7800 0x55bce573c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.422+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7c00 con 0x55bce57d7800 2024-04-27T05:22:33.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5e269640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bce57d7800 0x55bce573c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5e269640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bce57d7800 0x55bce573c100 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c61923880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c61923880 20 searching for the correct realm 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7000 con 0x55bce57d7800 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5e269640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bce57d7000 0x55bce573cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7c00 con 0x55bce57d7000 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.423+0000 7f7c5da68640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bce57d7000 0x55bce573cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.424+0000 7f7c5da68640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bce57d7000 0x55bce573cc00 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.424+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.424+0000 7f7c5da68640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bce57d7c00 0x55bce573d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.424+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.424+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bce57d7c00 0x55bce573d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.424+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bce57d7c00 0x55bce573d180 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.425+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7000 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.426+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7000 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.427+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.428+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7000 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.429+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7000 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7800 2024-04-27T05:22:33.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.430+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7c00 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce6251000 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d6c00 con 0x55bce57d7000 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7400 con 0x55bce6251000 2024-04-27T05:22:33.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.431+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce6251400 con 0x55bce6251000 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce6251800 con 0x55bce57d7800 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 zone default found 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 4 Realm: () 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 zonegroup default 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800000 con 0x55bce57d7c00 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800400 con 0x55bce57d7800 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.432+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800800 con 0x55bce57d7800 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c61923880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c61923880 20 started sync module instance, tier type = 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c61923880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.433+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800c00 con 0x55bce57d7000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.434+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.434+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94269791742208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801000 con 0x55bce57d7000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.434+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801400 con 0x55bce6251000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.435+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94269791742208] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.435+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.435+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94269791743360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801800 con 0x55bce6251000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.435+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801c00 con 0x55bce57d7800 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.436+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.436+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94269791744512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800400 con 0x55bce57d7800 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.436+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800800 con 0x55bce57d7000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.436+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94269791743360] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.437+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94269791744512] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.437+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.437+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94269791745664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801c00 con 0x55bce57d7000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.437+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800000 con 0x55bce6251000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.438+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94269791745664] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.438+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.438+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94269791746816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce6251800 con 0x55bce6251000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.438+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce6251400 con 0x55bce57d7c00 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.439+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.439+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94269791746816] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.439+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94269791747968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce57d7400 con 0x55bce57d7c00 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.439+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce63b7800 con 0x55bce57d7c00 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.440+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94269791747968] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.441+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.441+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94269791749120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce63b7400 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.441+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800c00 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94269791749120] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94269791750272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801000 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=0 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=1 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=2 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=3 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=4 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=5 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.442+0000 7f7c61923880 20 add_watcher() i=6 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.443+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94269791750272] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.443+0000 7f7c61923880 20 add_watcher() i=7 2024-04-27T05:22:33.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.443+0000 7f7c61923880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.443+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5800800 con 0x55bce6251000 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.443+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.443+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94269791751424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce5801400 con 0x55bce6251000 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.444+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94269791751424] v31'210 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.444+0000 7f7c45237640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.444+0000 7f7c61923880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.444+0000 7f7c61923880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.444+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c61923880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c61923880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce547c8c0 con 0x55bce57d6c00 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c5da68640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c5da68640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35914/0 (socket says 172.21.15.138:35914) 2024-04-27T05:22:33.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c5da68640 1 -- 172.21.15.138:0/340309161 learned_addr learned my addr 172.21.15.138:0/340309161 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c5da68640 1 -- 172.21.15.138:0/340309161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce54ca680 con 0x55bce57d6c00 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.445+0000 7f7c5da68640 1 --2- 172.21.15.138:0/340309161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x55bce57bb620 tx=0x55bce63c8f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f95058e6323d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c44a36640 1 -- 172.21.15.138:0/340309161 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5802580 con 0x55bce57d6c00 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c44a36640 1 -- 172.21.15.138:0/340309161 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce6316000 con 0x55bce57d6c00 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c44a36640 1 -- 172.21.15.138:0/340309161 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce58026e0 con 0x55bce57d6c00 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 -- 172.21.15.138:0/340309161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 msgr2=0x55bce573c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 --2- 172.21.15.138:0/340309161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x55bce57bb620 tx=0x55bce63c8f00 comp rx=0 tx=0).stop 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 -- 172.21.15.138:0/340309161 shutdown_connections 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 --2- 172.21.15.138:0/340309161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 -- 172.21.15.138:0/340309161 >> 172.21.15.138:0/340309161 conn(0x55bce5801800 msgr2=0x55bce5563800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 -- 172.21.15.138:0/340309161 shutdown_connections 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.446+0000 7f7c61923880 1 -- 172.21.15.138:0/340309161 wait complete. 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c61923880 1 Processor -- start 2024-04-27T05:22:33.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c61923880 1 -- start start 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c61923880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c61923880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce547c780 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c5da68640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c5da68640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35926/0 (socket says 172.21.15.138:35926) 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c5da68640 1 -- 172.21.15.138:0/1572206697 learned_addr learned my addr 172.21.15.138:0/1572206697 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c5da68640 1 -- 172.21.15.138:0/1572206697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce54ca4e0 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c5da68640 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x55bce57bb830 tx=0x55bce63c9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c43233640 1 -- 172.21.15.138:0/1572206697 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5802840 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.447+0000 7f7c43233640 1 -- 172.21.15.138:0/1572206697 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce6316a80 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bce545d380 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c43233640 1 -- 172.21.15.138:0/1572206697 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bce5802580 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bce545cea0 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c43233640 1 -- 172.21.15.138:0/1572206697 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bce54838c0 con 0x55bce57d6c00 2024-04-27T05:22:33.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c43233640 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce63b7800 0x55bce573d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c43233640 1 -- 172.21.15.138:0/1572206697 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bce546bdc0 con 0x55bce57d6c00 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c61923880 5 note: GC not initialized 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.448+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce63b7800 0x55bce573d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c40a2e640 20 reqs_thread_entry: start 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c61923880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c5ea6a640 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce63b7800 0x55bce573d700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55bce63c66c0 tx=0x55bce63c9e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c61923880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c3ea2a640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c3e208640 20 reqs_thread_entry: start 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c61923880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2000 con 0x55bce6251000 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c61923880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.449+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2400 con 0x55bce57d7c00 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.450+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.450+0000 7f7c45237640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.450+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94269791751424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2800 con 0x55bce6251000 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.451+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94269791751424] v31'211 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.451+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94269791742208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2c00 con 0x55bce57d7000 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.452+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94269791742208] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.452+0000 7f7c61923880 20 remove_watcher() i=0 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.452+0000 7f7c61923880 2 removed watcher, disabling cache 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.452+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94269791743360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d3000 con 0x55bce6251000 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.454+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94269791743360] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.454+0000 7f7c61923880 20 remove_watcher() i=1 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.454+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94269791744512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d3400 con 0x55bce57d7800 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.455+0000 7f7c5e269640 1 -- 172.21.15.138:0/2969897650 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94269791744512] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef200 con 0x55bce57d7800 2024-04-27T05:22:33.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.455+0000 7f7c61923880 20 remove_watcher() i=2 2024-04-27T05:22:33.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.455+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94269791745664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d3800 con 0x55bce57d7000 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.456+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94269791745664] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57ef440 con 0x55bce57d7000 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.456+0000 7f7c61923880 20 remove_watcher() i=3 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.456+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94269791746816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d3c00 con 0x55bce6251000 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.457+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94269791746816] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce6251000 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.457+0000 7f7c61923880 20 remove_watcher() i=4 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.457+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94269791747968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2000 con 0x55bce57d7c00 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.457+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94269791747968] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.457+0000 7f7c61923880 20 remove_watcher() i=5 2024-04-27T05:22:33.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.457+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94269791749120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2400 con 0x55bce57d7c00 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.458+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94269791749120] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.458+0000 7f7c61923880 20 remove_watcher() i=6 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.458+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94269791750272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bce58d2800 con 0x55bce57d7c00 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c5ea6a640 1 -- 172.21.15.138:0/2969897650 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94269791750272] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bce57eeb40 con 0x55bce57d7c00 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 20 remove_watcher() i=7 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bce57d7c00 msgr2=0x55bce573d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bce57d7c00 0x55bce573d180 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bce6251000 msgr2=0x55bce573bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bce6251000 0x55bce573bb80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bce57d7000 msgr2=0x55bce573cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bce57d7000 0x55bce573cc00 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bce57d7800 msgr2=0x55bce573c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.459+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bce57d7800 0x55bce573c100 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce57d6000 msgr2=0x55bce573b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce57d6000 0x55bce573b600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55bce5565410 tx=0x55bce63c8f60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 msgr2=0x55bce573ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/2969897650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce6250c00 0x55bce573ab00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55bce6315410 tx=0x55bce556b980 comp rx=0 tx=0).stop 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 reap_dead start 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c5da68640 1 -- 172.21.15.138:0/2969897650 reap_dead start 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 shutdown_connections 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 >> 172.21.15.138:0/2969897650 conn(0x55bce5459400 msgr2=0x55bce5563000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 shutdown_connections 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/2969897650 wait complete. 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce63b7800 msgr2=0x55bce573d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce63b7800 0x55bce573d700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55bce63c66c0 tx=0x55bce63c9e60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 msgr2=0x55bce573c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x55bce57bb830 tx=0x55bce63c9080 comp rx=0 tx=0).stop 2024-04-27T05:22:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 shutdown_connections 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce63b7800 0x55bce573d700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/1572206697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce57d6c00 0x55bce573c680 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 >> 172.21.15.138:0/1572206697 conn(0x55bce5801800 msgr2=0x55bce5563800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 shutdown_connections 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/1572206697 wait complete. 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce6251c00 msgr2=0x55bce573b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce6251c00 0x55bce573b080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55bce63582d0 tx=0x55bce556bc20 comp rx=0 tx=0).stop 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 msgr2=0x55bce573a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55bce6314e10 tx=0x55bce556ac60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 shutdown_connections 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bce6251c00 0x55bce573b080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.460+0000 7f7c61923880 1 --2- 172.21.15.138:0/3905975617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bce5459000 0x55bce573a580 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.461+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 >> 172.21.15.138:0/3905975617 conn(0x55bce5458c00 msgr2=0x55bce5561000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.461+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 shutdown_connections 2024-04-27T05:22:33.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.461+0000 7f7c61923880 1 -- 172.21.15.138:0/3905975617 wait complete. 2024-04-27T05:22:33.477 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_obj", 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.479 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.480 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "put_obj", 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.481 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 30000042, 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 11, 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 11 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.482 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.483 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}}]} 2024-04-27T05:22:33.483 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-27T05:22:33.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-27T05:22:33.483 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-04-27T05:22:33.528 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.528 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f5639126880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b632c61400 0x55b632f42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f5639126880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b632f38600 con 0x55b632c61000 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f5639126880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b632ca0280 con 0x55b632c61400 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f56351c2640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b632c61400 0x55b632f42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f56359c3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55b632c61000 0x55b632d68800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40940/0 (socket says 172.21.15.138:40940) 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.526+0000 7f56359c3640 1 -- 172.21.15.138:0/3813925635 learned_addr learned my addr 172.21.15.138:0/3813925635 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.527+0000 7f56351c2640 1 -- 172.21.15.138:0/3813925635 >> v1:172.21.15.114:6789/0 conn(0x55b632c61000 legacy=0x55b632d68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.527+0000 7f56351c2640 1 -- 172.21.15.138:0/3813925635 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632c65380 con 0x55b632c61400 2024-04-27T05:22:33.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.527+0000 7f56349c1640 1 -- 172.21.15.138:0/3813925635 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443375534 0 0) 0x55b632c649c0 con 0x55b632c61000 2024-04-27T05:22:33.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.528+0000 7f56351c2640 1 --2- 172.21.15.138:0/3813925635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61400 0x55b632f42580 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55b632d6cc00 tx=0x55b632d72de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ba9d03064cb377 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.528+0000 7f56349c1640 1 -- 172.21.15.138:0/3813925635 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c79ce0 con 0x55b632c61400 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.528+0000 7f56349c1640 1 -- 172.21.15.138:0/3813925635 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b632f39b00 con 0x55b632c61400 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.528+0000 7f56349c1640 1 -- 172.21.15.138:0/3813925635 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c79e40 con 0x55b632c61400 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 -- 172.21.15.138:0/3813925635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61400 msgr2=0x55b632f42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 --2- 172.21.15.138:0/3813925635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61400 0x55b632f42580 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x55b632d6cc00 tx=0x55b632d72de0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 -- 172.21.15.138:0/3813925635 shutdown_connections 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 --2- 172.21.15.138:0/3813925635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61400 0x55b632f42580 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 -- 172.21.15.138:0/3813925635 >> 172.21.15.138:0/3813925635 conn(0x55b632c60c00 msgr2=0x55b632d68000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 -- 172.21.15.138:0/3813925635 shutdown_connections 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 -- 172.21.15.138:0/3813925635 wait complete. 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.529+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f5639126880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f5639126880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b632ca0780 con 0x55b632c61000 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56359c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56359c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35946/0 (socket says 172.21.15.138:35946) 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56359c3640 1 -- 172.21.15.138:0/2565830279 learned_addr learned my addr 172.21.15.138:0/2565830279 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56359c3640 1 -- 172.21.15.138:0/2565830279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632c64ea0 con 0x55b632c61000 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56359c3640 1 --2- 172.21.15.138:0/2565830279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55b633b1cc90 tx=0x55b632d72cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd4df17b9b38c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56341c0640 1 -- 172.21.15.138:0/2565830279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c794a0 con 0x55b632c61000 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56341c0640 1 -- 172.21.15.138:0/2565830279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b632f39e00 con 0x55b632c61000 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.530+0000 7f56341c0640 1 -- 172.21.15.138:0/2565830279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c79ce0 con 0x55b632c61000 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- 172.21.15.138:0/2565830279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 msgr2=0x55b632f42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 --2- 172.21.15.138:0/2565830279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55b633b1cc90 tx=0x55b632d72cc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- 172.21.15.138:0/2565830279 shutdown_connections 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 --2- 172.21.15.138:0/2565830279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- 172.21.15.138:0/2565830279 >> 172.21.15.138:0/2565830279 conn(0x55b632c60c00 msgr2=0x55b632d69000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- 172.21.15.138:0/2565830279 shutdown_connections 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- 172.21.15.138:0/2565830279 wait complete. 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f5639126880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b632ca08c0 con 0x55b632c61000 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f56359c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f56359c3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35958/0 (socket says 172.21.15.138:35958) 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.531+0000 7f56359c3640 1 -- 172.21.15.138:0/274738465 learned_addr learned my addr 172.21.15.138:0/274738465 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f56359c3640 1 -- 172.21.15.138:0/274738465 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632c656c0 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f56359c3640 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55b633b1d350 tx=0x55b632d72c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f56329bd640 1 -- 172.21.15.138:0/274738465 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c79e40 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f56329bd640 1 -- 172.21.15.138:0/274738465 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b633b38000 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b632c651e0 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f56329bd640 1 -- 172.21.15.138:0/274738465 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c794a0 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.532+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b632cd2ea0 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56329bd640 1 -- 172.21.15.138:0/274738465 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b632c8b600 con 0x55b632c61000 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f5639126880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f5639126880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b632ca0c80 con 0x55b633a59800 2024-04-27T05:22:33.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56329bd640 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633bba000 0x55b632f43080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56329bd640 1 -- 172.21.15.138:0/274738465 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55b632c73a40 con 0x55b632c61000 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56359c3640 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633bba000 0x55b632f43080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56361c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56361c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35964/0 (socket says 172.21.15.138:35964) 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56361c4640 1 -- 172.21.15.138:0/3891898625 learned_addr learned my addr 172.21.15.138:0/3891898625 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56361c4640 1 -- 172.21.15.138:0/3891898625 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632cd2d00 con 0x55b633a59800 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56359c3640 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633bba000 0x55b632f43080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55b633b6c420 tx=0x55b632d734a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.533+0000 7f56361c4640 1 --2- 172.21.15.138:0/3891898625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55b633b1d530 tx=0x55b632d73f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dea892d8a004cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f56301b8640 1 -- 172.21.15.138:0/3891898625 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c79e40 con 0x55b633a59800 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f56301b8640 1 -- 172.21.15.138:0/3891898625 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b633b38300 con 0x55b633a59800 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f56301b8640 1 -- 172.21.15.138:0/3891898625 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c794a0 con 0x55b633a59800 2024-04-27T05:22:33.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- 172.21.15.138:0/3891898625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 msgr2=0x55b632f42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 --2- 172.21.15.138:0/3891898625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55b633b1d530 tx=0x55b632d73f80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- 172.21.15.138:0/3891898625 shutdown_connections 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 --2- 172.21.15.138:0/3891898625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- 172.21.15.138:0/3891898625 >> 172.21.15.138:0/3891898625 conn(0x55b632c61400 msgr2=0x55b632d6b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- 172.21.15.138:0/3891898625 shutdown_connections 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- 172.21.15.138:0/3891898625 wait complete. 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.534+0000 7f5639126880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b632c84280 con 0x55b633a59800 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f56361c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f56361c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35970/0 (socket says 172.21.15.138:35970) 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 learned_addr learned my addr 172.21.15.138:0/1866228450 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632cd2b60 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f56361c4640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55b633b1d8f0 tx=0x55b632d73bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f562e9b5640 1 -- 172.21.15.138:0/1866228450 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c79e40 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f562e9b5640 1 -- 172.21.15.138:0/1866228450 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b633b38480 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b632cd29c0 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f562e9b5640 1 -- 172.21.15.138:0/1866228450 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b632c794a0 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.535+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b632cd2820 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f562e9b5640 1 -- 172.21.15.138:0/1866228450 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b632c8b8c0 con 0x55b633a59800 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f562e9b5640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b632ff2000 0x55b632f43600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f562e9b5640 1 -- 172.21.15.138:0/1866228450 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55b632c73c00 con 0x55b633a59800 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f56351c2640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b632ff2000 0x55b632f43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b633a59c00 0x55b632f43b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2400 con 0x55b633a59c00 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f56361c4640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b633a59c00 0x55b632f43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.536+0000 7f56351c2640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b632ff2000 0x55b632f43600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55b632d6cc00 tx=0x55b633bdef60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f56361c4640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b633a59c00 0x55b632f43b80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 realm 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2800 con 0x55b633a59c00 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2c00 con 0x55b633a59c00 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3000 con 0x55b633a59c00 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b632ff3400 0x55b632f44100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.537+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2400 con 0x55b632ff3400 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f56359c3640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b632ff3400 0x55b632f44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f56359c3640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b632ff3400 0x55b632f44100 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f5639126880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f5639126880 20 searching for the correct realm 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2800 con 0x55b632ff3400 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f56359c3640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b632ff2800 0x55b632f44c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.538+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2400 con 0x55b632ff2800 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56351c2640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b632ff2800 0x55b632f44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56351c2640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b632ff2800 0x55b632f44c00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56351c2640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b632ff2400 0x55b632f45180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56361c4640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b632ff2400 0x55b632f45180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.539+0000 7f56361c4640 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b632ff2400 0x55b632f45180 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.540+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2800 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.541+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2800 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.542+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.543+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.544+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.544+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.544+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.544+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.544+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2800 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.545+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2800 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff3400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.546+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b633a59c00 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a58800 con 0x55b632ff2800 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2c00 con 0x55b633a59c00 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3000 con 0x55b633a59c00 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.547+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3800 con 0x55b632ff3400 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 zone default found 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 4 Realm: () 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 zonegroup default 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3c00 con 0x55b632ff2400 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44000 con 0x55b632ff3400 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.548+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44400 con 0x55b632ff3400 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.549+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.549+0000 7f5639126880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.549+0000 7f5639126880 20 started sync module instance, tier type = 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.549+0000 7f5639126880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.549+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44800 con 0x55b632ff2800 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.550+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.550+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94241027238144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44c00 con 0x55b632ff2800 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.550+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a45000 con 0x55b633a59c00 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.551+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94241027238144] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.551+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.551+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94241027239296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a45400 con 0x55b633a59c00 2024-04-27T05:22:33.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.551+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a45800 con 0x55b632ff3400 2024-04-27T05:22:33.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.552+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94241027239296] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.552+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.552+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94241027240448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44000 con 0x55b632ff3400 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.552+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44400 con 0x55b632ff2800 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.553+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94241027240448] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.553+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.553+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94241027241600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a45800 con 0x55b632ff2800 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.553+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3800 con 0x55b633a59c00 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.554+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94241027241600] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.554+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.554+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94241027242752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff2c00 con 0x55b633a59c00 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.554+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3000 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.555+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.555+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94241027243904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b632ff3c00 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.555+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94241027242752] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.555+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633bbac00 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.556+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94241027243904] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.556+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.556+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94241027245056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44800 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.556+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44c00 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94241027245056] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94241027246208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a44400 con 0x55b632ff2400 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=0 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=1 2024-04-27T05:22:33.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=2 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=3 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=4 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=5 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.557+0000 7f5639126880 20 add_watcher() i=6 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.558+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94241027246208] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.558+0000 7f5639126880 20 add_watcher() i=7 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.558+0000 7f5639126880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.558+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a45800 con 0x55b633a59c00 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.558+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.558+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94241027247360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b633a45000 con 0x55b633a59c00 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.559+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94241027247360] v31'212 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.559+0000 7f561c991640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.559+0000 7f5639126880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.559+0000 7f5639126880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f5639126880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f5639126880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b632c848c0 con 0x55b633a45c00 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f56351c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f56351c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35986/0 (socket says 172.21.15.138:35986) 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f56351c2640 1 -- 172.21.15.138:0/3593412193 learned_addr learned my addr 172.21.15.138:0/3593412193 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f56351c2640 1 -- 172.21.15.138:0/3593412193 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632cd2680 con 0x55b633a45c00 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f56351c2640 1 --2- 172.21.15.138:0/3593412193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55b632fd2c30 tx=0x55b633bdefc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e9b7b3e4b9f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f561c190640 1 -- 172.21.15.138:0/3593412193 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b633a46580 con 0x55b633a45c00 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f561c190640 1 -- 172.21.15.138:0/3593412193 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b633b38000 con 0x55b633a45c00 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.560+0000 7f561c190640 1 -- 172.21.15.138:0/3593412193 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b633a466e0 con 0x55b633a45c00 2024-04-27T05:22:33.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 -- 172.21.15.138:0/3593412193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 msgr2=0x55b632f44680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 --2- 172.21.15.138:0/3593412193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55b632fd2c30 tx=0x55b633bdefc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 -- 172.21.15.138:0/3593412193 shutdown_connections 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 --2- 172.21.15.138:0/3593412193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 -- 172.21.15.138:0/3593412193 >> 172.21.15.138:0/3593412193 conn(0x55b633a45400 msgr2=0x55b632d6b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 -- 172.21.15.138:0/3593412193 shutdown_connections 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 -- 172.21.15.138:0/3593412193 wait complete. 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 Processor -- start 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.561+0000 7f5639126880 1 -- start start 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f5639126880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f5639126880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b632c84780 con 0x55b633a45c00 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f56351c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f56351c2640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35994/0 (socket says 172.21.15.138:35994) 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f56351c2640 1 -- 172.21.15.138:0/3970492474 learned_addr learned my addr 172.21.15.138:0/3970492474 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f56351c2640 1 -- 172.21.15.138:0/3970492474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b632c64ea0 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f56351c2640 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55b632fd37a0 tx=0x55b633bdee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f561a98d640 1 -- 172.21.15.138:0/3970492474 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b633a46840 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f561a98d640 1 -- 172.21.15.138:0/3970492474 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b633b38780 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b632c656c0 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f561a98d640 1 -- 172.21.15.138:0/3970492474 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b633a46580 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.562+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b632c651e0 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.563+0000 7f561a98d640 1 -- 172.21.15.138:0/3970492474 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b632c8ab00 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.563+0000 7f561a98d640 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633a44800 0x55b632f45700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.563+0000 7f561a98d640 1 -- 172.21.15.138:0/3970492474 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55b632c73dc0 con 0x55b633a45c00 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.563+0000 7f5639126880 5 note: GC not initialized 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.563+0000 7f56361c4640 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633a44800 0x55b632f45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.563+0000 7f5618188640 20 reqs_thread_entry: start 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5639126880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f56361c4640 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633a44800 0x55b632f45700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55b633be0a20 tx=0x55b6330b8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5639126880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5616184640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5615962640 20 reqs_thread_entry: start 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5639126880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8000 con 0x55b633a59c00 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5639126880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.564+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8400 con 0x55b632ff2400 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.565+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.566+0000 7f561c991640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.566+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94241027247360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8800 con 0x55b633a59c00 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.567+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94241027247360] v31'213 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.567+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94241027238144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8c00 con 0x55b632ff2800 2024-04-27T05:22:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.568+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94241027238144] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.568+0000 7f5639126880 20 remove_watcher() i=0 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.568+0000 7f5639126880 2 removed watcher, disabling cache 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.568+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94241027239296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d9000 con 0x55b633a59c00 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.569+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94241027239296] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.569+0000 7f5639126880 20 remove_watcher() i=1 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.569+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94241027240448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d9400 con 0x55b632ff3400 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.570+0000 7f56359c3640 1 -- 172.21.15.138:0/1866228450 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94241027240448] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006fc0 con 0x55b632ff3400 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.570+0000 7f5639126880 20 remove_watcher() i=2 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.570+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94241027241600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d9800 con 0x55b632ff2800 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.571+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94241027241600] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633007200 con 0x55b632ff2800 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.571+0000 7f5639126880 20 remove_watcher() i=3 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.571+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94241027242752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d9c00 con 0x55b633a59c00 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.572+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94241027242752] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b633a59c00 2024-04-27T05:22:33.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.572+0000 7f5639126880 20 remove_watcher() i=4 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.572+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94241027243904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8000 con 0x55b632ff2400 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.573+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94241027243904] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.573+0000 7f5639126880 20 remove_watcher() i=5 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.573+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94241027245056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8400 con 0x55b632ff2400 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.574+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94241027245056] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.574+0000 7f5639126880 20 remove_watcher() i=6 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.574+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94241027246208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6330d8800 con 0x55b632ff2400 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f56361c4640 1 -- 172.21.15.138:0/1866228450 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94241027246208] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b633006900 con 0x55b632ff2400 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 20 remove_watcher() i=7 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b632ff2400 msgr2=0x55b632f45180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b632ff2400 0x55b632f45180 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b633a59c00 msgr2=0x55b632f43b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b633a59c00 0x55b632f43b80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b632ff2800 msgr2=0x55b632f44c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b632ff2800 0x55b632f44c00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b632ff3400 msgr2=0x55b632f44100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b632ff3400 0x55b632f44100 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b632ff2000 msgr2=0x55b632f43600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b632ff2000 0x55b632f43600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55b632d6cc00 tx=0x55b633bdef60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 msgr2=0x55b632f42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/1866228450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a59800 0x55b632f42b00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55b633b1d8f0 tx=0x55b632d73bc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 reap_dead start 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 shutdown_connections 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 >> 172.21.15.138:0/1866228450 conn(0x55b632c61400 msgr2=0x55b632d6b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f56351c2640 1 -- 172.21.15.138:0/1866228450 reap_dead start 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 shutdown_connections 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/1866228450 wait complete. 2024-04-27T05:22:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633a44800 msgr2=0x55b632f45700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633a44800 0x55b632f45700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55b633be0a20 tx=0x55b6330b8000 comp rx=0 tx=0).stop 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 msgr2=0x55b632f44680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55b632fd37a0 tx=0x55b633bdee40 comp rx=0 tx=0).stop 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 shutdown_connections 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633a44800 0x55b632f45700 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 --2- 172.21.15.138:0/3970492474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b633a45c00 0x55b632f44680 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 >> 172.21.15.138:0/3970492474 conn(0x55b633a45400 msgr2=0x55b632d6b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 shutdown_connections 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.575+0000 7f5639126880 1 -- 172.21.15.138:0/3970492474 wait complete. 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633bba000 msgr2=0x55b632f43080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633bba000 0x55b632f43080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55b633b6c420 tx=0x55b632d734a0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 msgr2=0x55b632f42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55b633b1d350 tx=0x55b632d72c60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 shutdown_connections 2024-04-27T05:22:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b633bba000 0x55b632f43080 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 --2- 172.21.15.138:0/274738465 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b632c61000 0x55b632f42580 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 >> 172.21.15.138:0/274738465 conn(0x55b632c60c00 msgr2=0x55b632d69000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 shutdown_connections 2024-04-27T05:22:33.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.576+0000 7f5639126880 1 -- 172.21.15.138:0/274738465 wait complete. 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.604 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:33.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_obj", 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.606 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 4, 2024-04-27T05:22:33.607 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.607 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.607 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.607 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T05:22:33.607 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-27T05:22:33.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-27T05:22:33.607 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-04-27T05:22:33.652 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.652 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.650+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.650+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34f742880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x558bff751000 0x558bffa32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34f742880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558bffa28600 con 0x558bff751400 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34f742880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x558bff790280 con 0x558bff751000 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34c148640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x558bff751000 0x558bffa32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34b947640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x558bff751400 0x558bff858800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40954/0 (socket says 172.21.15.138:40954) 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34b947640 1 -- 172.21.15.138:0/2097219964 learned_addr learned my addr 172.21.15.138:0/2097219964 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34c148640 1 -- 172.21.15.138:0/2097219964 >> v1:172.21.15.114:6789/0 conn(0x558bff751400 legacy=0x558bff858800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.651+0000 7fd34c148640 1 -- 172.21.15.138:0/2097219964 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff755380 con 0x558bff751000 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34c148640 1 --2- 172.21.15.138:0/2097219964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x558bff85cd50 tx=0x558bff862de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e4734a530a301b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34b146640 1 -- 172.21.15.138:0/2097219964 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff769ce0 con 0x558bff751000 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34b146640 1 -- 172.21.15.138:0/2097219964 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bffa29b00 con 0x558bff751000 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34b146640 1 -- 172.21.15.138:0/2097219964 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff769e40 con 0x558bff751000 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 -- 172.21.15.138:0/2097219964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 msgr2=0x558bffa32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 --2- 172.21.15.138:0/2097219964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x558bff85cd50 tx=0x558bff862de0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 -- 172.21.15.138:0/2097219964 shutdown_connections 2024-04-27T05:22:33.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 --2- 172.21.15.138:0/2097219964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 -- 172.21.15.138:0/2097219964 >> 172.21.15.138:0/2097219964 conn(0x558bff750c00 msgr2=0x558bff858000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 -- 172.21.15.138:0/2097219964 shutdown_connections 2024-04-27T05:22:33.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.653+0000 7fd34f742880 1 -- 172.21.15.138:0/2097219964 wait complete. 2024-04-27T05:22:33.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.654+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.654+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.654+0000 7fd34f742880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.654+0000 7fd34f742880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff790780 con 0x558bff751000 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.655+0000 7fd34c148640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.655+0000 7fd34c148640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36008/0 (socket says 172.21.15.138:36008) 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.655+0000 7fd34c148640 1 -- 172.21.15.138:0/2315940982 learned_addr learned my addr 172.21.15.138:0/2315940982 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.655+0000 7fd34c148640 1 -- 172.21.15.138:0/2315940982 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff754ea0 con 0x558bff751000 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34c148640 1 --2- 172.21.15.138:0/2315940982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x558c00551560 tx=0x558bff862cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ea6473db2acce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34a945640 1 -- 172.21.15.138:0/2315940982 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff7694a0 con 0x558bff751000 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34a945640 1 -- 172.21.15.138:0/2315940982 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bffa29e00 con 0x558bff751000 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34a945640 1 -- 172.21.15.138:0/2315940982 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff769ce0 con 0x558bff751000 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 -- 172.21.15.138:0/2315940982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 msgr2=0x558bffa32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 --2- 172.21.15.138:0/2315940982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x558c00551560 tx=0x558bff862cc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 -- 172.21.15.138:0/2315940982 shutdown_connections 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 --2- 172.21.15.138:0/2315940982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 -- 172.21.15.138:0/2315940982 >> 172.21.15.138:0/2315940982 conn(0x558bff750c00 msgr2=0x558bff859000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 -- 172.21.15.138:0/2315940982 shutdown_connections 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 -- 172.21.15.138:0/2315940982 wait complete. 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.656+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34f742880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34f742880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff7908c0 con 0x558bff751000 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34c148640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34c148640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36014/0 (socket says 172.21.15.138:36014) 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34c148640 1 -- 172.21.15.138:0/2143742222 learned_addr learned my addr 172.21.15.138:0/2143742222 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34c148640 1 -- 172.21.15.138:0/2143742222 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff7556c0 con 0x558bff751000 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34c148640 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x558c00600de0 tx=0x558bff862c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd349142640 1 -- 172.21.15.138:0/2143742222 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff769e40 con 0x558bff751000 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd349142640 1 -- 172.21.15.138:0/2143742222 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c00536000 con 0x558bff751000 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558bff7551e0 con 0x558bff751000 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd349142640 1 -- 172.21.15.138:0/2143742222 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff7694a0 con 0x558bff751000 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.657+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bff7c2ea0 con 0x558bff751000 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34f742880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34f742880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff790c80 con 0x558c00544c00 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd349142640 1 -- 172.21.15.138:0/2143742222 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558bff77ab00 con 0x558bff751000 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34c949640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd349142640 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c00545c00 0x558bffa33080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34c949640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36020/0 (socket says 172.21.15.138:36020) 2024-04-27T05:22:33.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34c949640 1 -- 172.21.15.138:0/3964675305 learned_addr learned my addr 172.21.15.138:0/3964675305 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd349142640 1 -- 172.21.15.138:0/2143742222 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x558bff763a40 con 0x558bff751000 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.658+0000 7fd34c148640 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c00545c00 0x558bffa33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34c949640 1 -- 172.21.15.138:0/3964675305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff7c2d00 con 0x558c00544c00 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34c949640 1 --2- 172.21.15.138:0/3964675305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x558c00601b90 tx=0x558bff863f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01c223cadaa9301 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34c148640 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c00545c00 0x558bffa33080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x558c0064c2a0 tx=0x558bff863c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34693d640 1 -- 172.21.15.138:0/3964675305 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff769e40 con 0x558c00544c00 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34693d640 1 -- 172.21.15.138:0/3964675305 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c00536600 con 0x558c00544c00 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34693d640 1 -- 172.21.15.138:0/3964675305 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff7694a0 con 0x558c00544c00 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 -- 172.21.15.138:0/3964675305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 msgr2=0x558bffa32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 --2- 172.21.15.138:0/3964675305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x558c00601b90 tx=0x558bff863f80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 -- 172.21.15.138:0/3964675305 shutdown_connections 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 --2- 172.21.15.138:0/3964675305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 -- 172.21.15.138:0/3964675305 >> 172.21.15.138:0/3964675305 conn(0x558bff751400 msgr2=0x558bff85b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 -- 172.21.15.138:0/3964675305 shutdown_connections 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.659+0000 7fd34f742880 1 -- 172.21.15.138:0/3964675305 wait complete. 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34f742880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34f742880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff774280 con 0x558c00544c00 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34c949640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34c949640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36022/0 (socket says 172.21.15.138:36022) 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 learned_addr learned my addr 172.21.15.138:0/2552013845 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff7c2b60 con 0x558c00544c00 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34c949640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x558c005515c0 tx=0x558bff863980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34513a640 1 -- 172.21.15.138:0/2552013845 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff769e40 con 0x558c00544c00 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34513a640 1 -- 172.21.15.138:0/2552013845 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c00536780 con 0x558c00544c00 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.660+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558bff7c29c0 con 0x558c00544c00 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34513a640 1 -- 172.21.15.138:0/2552013845 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bff7694a0 con 0x558c00544c00 2024-04-27T05:22:33.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bff7c2820 con 0x558c00544c00 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34513a640 1 -- 172.21.15.138:0/2552013845 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558bff77b600 con 0x558c00544c00 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34513a640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558bffac6000 0x558bffa33600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34513a640 1 -- 172.21.15.138:0/2552013845 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x558bff763c00 con 0x558c00544c00 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34b947640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558bffac6000 0x558bffa33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558c00545000 0x558bffa33b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.661+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c00545400 con 0x558c00545000 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34c949640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558c00545000 0x558bffa33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34b947640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558bffac6000 0x558bffa33600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x558c006b3260 tx=0x558c0069aea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34c949640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558c00545000 0x558bffa33b80 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 realm 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c00545800 con 0x558c00545000 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7000 con 0x558c00545000 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.662+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7400 con 0x558c00545000 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558bffac7800 0x558bffa34100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7c00 con 0x558bffac7800 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34c148640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558bffac7800 0x558bffa34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34c148640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558bffac7800 0x558bffa34100 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 20 searching for the correct realm 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.663+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7000 con 0x558bffac7800 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34c148640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558bffac7000 0x558bffa34c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7c00 con 0x558bffac7000 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34b947640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558bffac7000 0x558bffa34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34b947640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558bffac7000 0x558bffa34c00 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34b947640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558bffac7c00 0x558bffa35180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.664+0000 7fd34c949640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558bffac7c00 0x558bffa35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558bffac7c00 0x558bffa35180 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.665+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.666+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.667+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.668+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7000 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.669+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.670+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.670+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.670+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.670+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.670+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.670+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7000 2024-04-27T05:22:33.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7800 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7c00 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558c00545000 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.671+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac6c00 con 0x558bffac7000 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7400 con 0x558c00545000 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c00545400 con 0x558c00545000 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c00545800 con 0x558bffac7800 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 zone default found 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 4 Realm: () 2024-04-27T05:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 zonegroup default 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.672+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2000 con 0x558bffac7c00 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2400 con 0x558bffac7800 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2800 con 0x558bffac7800 2024-04-27T05:22:33.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 started sync module instance, tier type = 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.673+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2c00 con 0x558bffac7000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.674+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.674+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94059777665280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3000 con 0x558bffac7000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.674+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3400 con 0x558c00545000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.675+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94059777665280] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.676+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.676+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94059777666432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3800 con 0x558c00545000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.676+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3c00 con 0x558bffac7800 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.677+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94059777666432] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.677+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.677+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94059777667584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2400 con 0x558bffac7800 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.677+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2800 con 0x558bffac7000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.678+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94059777667584] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.678+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.678+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94059777668736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3c00 con 0x558bffac7000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.678+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2000 con 0x558c00545000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.679+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.679+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94059777669888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c00545800 con 0x558c00545000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.679+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c00545400 con 0x558bffac7c00 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.679+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94059777668736] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.680+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.680+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94059777671040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffac7400 con 0x558bffac7c00 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.680+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94059777669888] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.680+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c006ab800 con 0x558bffac7c00 2024-04-27T05:22:33.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.681+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94059777671040] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.681+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.681+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94059777672192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558c006ab400 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.681+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2c00 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94059777672192] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94059777673344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3000 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=0 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=1 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=2 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=3 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=4 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=5 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.682+0000 7fd34f742880 20 add_watcher() i=6 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.683+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94059777673344] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.683+0000 7fd34f742880 20 add_watcher() i=7 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.683+0000 7fd34f742880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.683+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf2800 con 0x558c00545000 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.683+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.683+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94059777674496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffaf3400 con 0x558c00545000 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.684+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94059777674496] v31'214 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.684+0000 7fd333116640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.684+0000 7fd34f742880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.684+0000 7fd34f742880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.684+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.684+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34f742880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34f742880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff7748c0 con 0x558bffac6c00 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34b947640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34b947640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36036/0 (socket says 172.21.15.138:36036) 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34b947640 1 -- 172.21.15.138:0/2088907488 learned_addr learned my addr 172.21.15.138:0/2088907488 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34b947640 1 -- 172.21.15.138:0/2088907488 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff7c2680 con 0x558bffac6c00 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd34b947640 1 --2- 172.21.15.138:0/2088907488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x558bffaa8930 tx=0x558c0069ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5529bc1ef98b762 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.685+0000 7fd332915640 1 -- 172.21.15.138:0/2088907488 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bffaf4580 con 0x558bffac6c00 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd332915640 1 -- 172.21.15.138:0/2088907488 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c00536000 con 0x558bffac6c00 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd332915640 1 -- 172.21.15.138:0/2088907488 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bffaf46e0 con 0x558bffac6c00 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 -- 172.21.15.138:0/2088907488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 msgr2=0x558bffa34680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 --2- 172.21.15.138:0/2088907488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x558bffaa8930 tx=0x558c0069ae40 comp rx=0 tx=0).stop 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 -- 172.21.15.138:0/2088907488 shutdown_connections 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 --2- 172.21.15.138:0/2088907488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 -- 172.21.15.138:0/2088907488 >> 172.21.15.138:0/2088907488 conn(0x558bffaf3800 msgr2=0x558bff85b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 -- 172.21.15.138:0/2088907488 shutdown_connections 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 -- 172.21.15.138:0/2088907488 wait complete. 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.686+0000 7fd34f742880 1 Processor -- start 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34f742880 1 -- start start 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34f742880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34f742880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bff774780 con 0x558bffac6c00 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34b947640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34b947640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36042/0 (socket says 172.21.15.138:36042) 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34b947640 1 -- 172.21.15.138:0/4235322175 learned_addr learned my addr 172.21.15.138:0/4235322175 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34b947640 1 -- 172.21.15.138:0/4235322175 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bff7c24e0 con 0x558bffac6c00 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34b947640 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x558bffaa9890 tx=0x558c0069afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd331112640 1 -- 172.21.15.138:0/4235322175 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bffaf4840 con 0x558bffac6c00 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd331112640 1 -- 172.21.15.138:0/4235322175 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c00536a80 con 0x558bffac6c00 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558bff755380 con 0x558bffac6c00 2024-04-27T05:22:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.687+0000 7fd331112640 1 -- 172.21.15.138:0/4235322175 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558bffaf4580 con 0x558bffac6c00 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.688+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bff754ea0 con 0x558bffac6c00 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.688+0000 7fd331112640 1 -- 172.21.15.138:0/4235322175 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x558bff77bb80 con 0x558bffac6c00 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.688+0000 7fd331112640 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c006ab800 0x558bffa35700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.688+0000 7fd331112640 1 -- 172.21.15.138:0/4235322175 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x558bff763dc0 con 0x558bffac6c00 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.688+0000 7fd34f742880 5 note: GC not initialized 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.688+0000 7fd34c949640 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c006ab800 0x558bffa35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd32e90d640 20 reqs_thread_entry: start 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34f742880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34f742880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34c949640 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c006ab800 0x558bffa35700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x558bffaa9230 tx=0x558c0069bda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd32c909640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:03 2024 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd32c0e7640 20 reqs_thread_entry: start 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34f742880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8000 con 0x558c00545000 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34f742880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.689+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8400 con 0x558bffac7c00 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.690+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.690+0000 7fd333116640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.690+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94059777674496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8800 con 0x558c00545000 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.691+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94059777674496] v31'215 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.691+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94059777665280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8c00 con 0x558bffac7000 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.692+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94059777665280] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.692+0000 7fd34f742880 20 remove_watcher() i=0 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.692+0000 7fd34f742880 2 removed watcher, disabling cache 2024-04-27T05:22:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.692+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94059777666432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc9000 con 0x558c00545000 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.694+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94059777666432] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.694+0000 7fd34f742880 20 remove_watcher() i=1 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.694+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94059777667584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc9400 con 0x558bffac7800 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.694+0000 7fd34c148640 1 -- 172.21.15.138:0/2552013845 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94059777667584] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0fc0 con 0x558bffac7800 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.694+0000 7fd34f742880 20 remove_watcher() i=2 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.694+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94059777668736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc9800 con 0x558bffac7000 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.695+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94059777668736] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae1200 con 0x558bffac7000 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.695+0000 7fd34f742880 20 remove_watcher() i=3 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.695+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94059777669888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc9c00 con 0x558c00545000 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.696+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94059777669888] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558c00545000 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.696+0000 7fd34f742880 20 remove_watcher() i=4 2024-04-27T05:22:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.696+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94059777671040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8000 con 0x558bffac7c00 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.697+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94059777671040] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.697+0000 7fd34f742880 20 remove_watcher() i=5 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.697+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94059777672192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8400 con 0x558bffac7c00 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.698+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94059777672192] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.698+0000 7fd34f742880 20 remove_watcher() i=6 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.698+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94059777673344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558bffbc8800 con 0x558bffac7c00 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34c949640 1 -- 172.21.15.138:0/2552013845 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94059777673344] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558bffae0900 con 0x558bffac7c00 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 20 remove_watcher() i=7 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558bffac7c00 msgr2=0x558bffa35180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x558bffac7c00 0x558bffa35180 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558c00545000 msgr2=0x558bffa33b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x558c00545000 0x558bffa33b80 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558bffac7000 msgr2=0x558bffa34c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x558bffac7000 0x558bffa34c00 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558bffac7800 msgr2=0x558bffa34100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.699+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x558bffac7800 0x558bffa34100 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558bffac6000 msgr2=0x558bffa33600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558bffac6000 0x558bffa33600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x558c006b3260 tx=0x558c0069aea0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 msgr2=0x558bffa32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/2552013845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558c00544c00 0x558bffa32b00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x558c005515c0 tx=0x558bff863980 comp rx=0 tx=0).stop 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 reap_dead start 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34b947640 1 -- 172.21.15.138:0/2552013845 reap_dead start 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 shutdown_connections 2024-04-27T05:22:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 >> 172.21.15.138:0/2552013845 conn(0x558bff751400 msgr2=0x558bff85b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 shutdown_connections 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2552013845 wait complete. 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c006ab800 msgr2=0x558bffa35700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c006ab800 0x558bffa35700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x558bffaa9230 tx=0x558c0069bda0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 msgr2=0x558bffa34680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x558bffaa9890 tx=0x558c0069afc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 shutdown_connections 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c006ab800 0x558bffa35700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/4235322175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bffac6c00 0x558bffa34680 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 >> 172.21.15.138:0/4235322175 conn(0x558bffaf3800 msgr2=0x558bff85b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 shutdown_connections 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/4235322175 wait complete. 2024-04-27T05:22:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c00545c00 msgr2=0x558bffa33080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c00545c00 0x558bffa33080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x558c0064c2a0 tx=0x558bff863c20 comp rx=0 tx=0).stop 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 msgr2=0x558bffa32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.700+0000 7fd34f742880 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x558c00600de0 tx=0x558bff862c60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.701+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 shutdown_connections 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.701+0000 7fd34f742880 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x558c00545c00 0x558bffa33080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.701+0000 7fd34f742880 1 --2- 172.21.15.138:0/2143742222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558bff751000 0x558bffa32580 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.701+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 >> 172.21.15.138:0/2143742222 conn(0x558bff750c00 msgr2=0x558bff859000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.701+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 shutdown_connections 2024-04-27T05:22:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.701+0000 7fd34f742880 1 -- 172.21.15.138:0/2143742222 wait complete. 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [ 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: "buckets": [ 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "", 2024-04-27T05:22:33.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [] 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo", 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.720 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 2, 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo2", 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.721 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo3", 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket": "myfoo4", 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-04-27T05:00:00.000000Z", 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1714194000, 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "owner": "foo", 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 0, 2024-04-27T05:22:33.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 1, 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 1 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [ 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "foo", 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "categories": [ 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "category": "delete_bucket", 2024-04-27T05:22:33.724 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "total": { 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_sent": 236, 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_received": 0, 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": 5, 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: "successful_ops": 4 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:22:33.725 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.726 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}}]} 2024-04-27T05:22:33.726 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-27T05:22:33.726 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-27T05:22:33.726 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-04-27T05:22:33.776 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.776 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa729617880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55af27b7d400 0x55af27e5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa729617880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55af27e54600 con 0x55af27b7d000 2024-04-27T05:22:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa729617880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55af27bbc280 con 0x55af27b7d400 2024-04-27T05:22:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa7252fa640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55af27b7d400 0x55af27e5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa725afb640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55af27b7d000 0x55af27c84800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40956/0 (socket says 172.21.15.138:40956) 2024-04-27T05:22:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.774+0000 7fa725afb640 1 -- 172.21.15.138:0/4020712535 learned_addr learned my addr 172.21.15.138:0/4020712535 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.775+0000 7fa7252fa640 1 -- 172.21.15.138:0/4020712535 >> v1:172.21.15.114:6789/0 conn(0x55af27b7d000 legacy=0x55af27c84800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.775+0000 7fa7252fa640 1 -- 172.21.15.138:0/4020712535 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27b81380 con 0x55af27b7d400 2024-04-27T05:22:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.775+0000 7fa724af9640 1 -- 172.21.15.138:0/4020712535 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484062250 0 0) 0x55af27b809c0 con 0x55af27b7d000 2024-04-27T05:22:33.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.776+0000 7fa7252fa640 1 --2- 172.21.15.138:0/4020712535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d400 0x55af27e5e580 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x55af27c88c00 tx=0x55af27c8ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d396c48950470cc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.776+0000 7fa724af9640 1 -- 172.21.15.138:0/4020712535 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b95ce0 con 0x55af27b7d400 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa724af9640 1 -- 172.21.15.138:0/4020712535 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af27e55b00 con 0x55af27b7d400 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa724af9640 1 -- 172.21.15.138:0/4020712535 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b95e40 con 0x55af27b7d400 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 -- 172.21.15.138:0/4020712535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d400 msgr2=0x55af27e5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 --2- 172.21.15.138:0/4020712535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d400 0x55af27e5e580 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x55af27c88c00 tx=0x55af27c8ede0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 -- 172.21.15.138:0/4020712535 shutdown_connections 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 --2- 172.21.15.138:0/4020712535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d400 0x55af27e5e580 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 -- 172.21.15.138:0/4020712535 >> 172.21.15.138:0/4020712535 conn(0x55af27b7cc00 msgr2=0x55af27c84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 -- 172.21.15.138:0/4020712535 shutdown_connections 2024-04-27T05:22:33.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.777+0000 7fa729617880 1 -- 172.21.15.138:0/4020712535 wait complete. 2024-04-27T05:22:33.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa729617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa729617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af27bbc780 con 0x55af27b7d000 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa725afb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa725afb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36062/0 (socket says 172.21.15.138:36062) 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa725afb640 1 -- 172.21.15.138:0/2995782522 learned_addr learned my addr 172.21.15.138:0/2995782522 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa725afb640 1 -- 172.21.15.138:0/2995782522 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27b80ea0 con 0x55af27b7d000 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa725afb640 1 --2- 172.21.15.138:0/2995782522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55af28a38c90 tx=0x55af27c8ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a345a7fcd71b7927 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.778+0000 7fa7242f8640 1 -- 172.21.15.138:0/2995782522 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b954a0 con 0x55af27b7d000 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa7242f8640 1 -- 172.21.15.138:0/2995782522 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af27e55e00 con 0x55af27b7d000 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa7242f8640 1 -- 172.21.15.138:0/2995782522 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b95ce0 con 0x55af27b7d000 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- 172.21.15.138:0/2995782522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 msgr2=0x55af27e5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 --2- 172.21.15.138:0/2995782522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55af28a38c90 tx=0x55af27c8ecc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- 172.21.15.138:0/2995782522 shutdown_connections 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 --2- 172.21.15.138:0/2995782522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- 172.21.15.138:0/2995782522 >> 172.21.15.138:0/2995782522 conn(0x55af27b7cc00 msgr2=0x55af27c85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- 172.21.15.138:0/2995782522 shutdown_connections 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- 172.21.15.138:0/2995782522 wait complete. 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.779+0000 7fa729617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af27bbc8c0 con 0x55af27b7d000 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa725afb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa725afb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36066/0 (socket says 172.21.15.138:36066) 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa725afb640 1 -- 172.21.15.138:0/1063526792 learned_addr learned my addr 172.21.15.138:0/1063526792 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa725afb640 1 -- 172.21.15.138:0/1063526792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27b816c0 con 0x55af27b7d000 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa725afb640 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55af28a39350 tx=0x55af27c8ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa722af5640 1 -- 172.21.15.138:0/1063526792 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b95e40 con 0x55af27b7d000 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa722af5640 1 -- 172.21.15.138:0/1063526792 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af28a4e000 con 0x55af27b7d000 2024-04-27T05:22:33.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa722af5640 1 -- 172.21.15.138:0/1063526792 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b954a0 con 0x55af27b7d000 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af27b811e0 con 0x55af27b7d000 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.780+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af27beeea0 con 0x55af27b7d000 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa722af5640 1 -- 172.21.15.138:0/1063526792 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55af27ba7600 con 0x55af27b7d000 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa729617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa729617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af27ba0280 con 0x55af28975800 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa722af5640 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28ad0000 0x55af27e5f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa722af5640 1 -- 172.21.15.138:0/1063526792 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55af27b8fa40 con 0x55af27b7d000 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa7262fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa7262fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36068/0 (socket says 172.21.15.138:36068) 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa7262fc640 1 -- 172.21.15.138:0/3814267832 learned_addr learned my addr 172.21.15.138:0/3814267832 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa725afb640 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28ad0000 0x55af27e5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa7262fc640 1 -- 172.21.15.138:0/3814267832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27beed00 con 0x55af28975800 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa7262fc640 1 --2- 172.21.15.138:0/3814267832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55af28a39e90 tx=0x55af27c8fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a1df4f8011ed00 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa725afb640 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28ad0000 0x55af27e5f080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55af28a82420 tx=0x55af27c8f4a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.781+0000 7fa7202f0640 1 -- 172.21.15.138:0/3814267832 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b95e40 con 0x55af28975800 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa7202f0640 1 -- 172.21.15.138:0/3814267832 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af28a4e300 con 0x55af28975800 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa7202f0640 1 -- 172.21.15.138:0/3814267832 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b954a0 con 0x55af28975800 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- 172.21.15.138:0/3814267832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 msgr2=0x55af27e5eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 --2- 172.21.15.138:0/3814267832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55af28a39e90 tx=0x55af27c8fc80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- 172.21.15.138:0/3814267832 shutdown_connections 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 --2- 172.21.15.138:0/3814267832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- 172.21.15.138:0/3814267832 >> 172.21.15.138:0/3814267832 conn(0x55af27b7d400 msgr2=0x55af27c87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- 172.21.15.138:0/3814267832 shutdown_connections 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- 172.21.15.138:0/3814267832 wait complete. 2024-04-27T05:22:33.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.782+0000 7fa729617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af27ba1180 con 0x55af28975800 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa7262fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa7262fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36082/0 (socket says 172.21.15.138:36082) 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 learned_addr learned my addr 172.21.15.138:0/1771356881 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27beeb60 con 0x55af28975800 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa7262fc640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55af28a399e0 tx=0x55af27c8f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa71eaed640 1 -- 172.21.15.138:0/1771356881 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b95e40 con 0x55af28975800 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa71eaed640 1 -- 172.21.15.138:0/1771356881 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af28a4e480 con 0x55af28975800 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af27bee9c0 con 0x55af28975800 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa71eaed640 1 -- 172.21.15.138:0/1771356881 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af27b954a0 con 0x55af28975800 2024-04-27T05:22:33.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.783+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af27bee820 con 0x55af28975800 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa71eaed640 1 -- 172.21.15.138:0/1771356881 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55af27ba7b80 con 0x55af28975800 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa71eaed640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af27f02000 0x55af27e5f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa71eaed640 1 -- 172.21.15.138:0/1771356881 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55af27b8fc00 con 0x55af28975800 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa7252fa640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af27f02000 0x55af27e5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55af28975c00 0x55af27e5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02400 con 0x55af28975c00 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa7262fc640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55af28975c00 0x55af27e5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.784+0000 7fa7252fa640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af27f02000 0x55af27e5f600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55af27c88c00 tx=0x55af28ad8d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa7262fc640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55af28975c00 0x55af27e5fb80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 realm 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02800 con 0x55af28975c00 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02c00 con 0x55af28975c00 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03000 con 0x55af28975c00 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55af27f03400 0x55af27e60100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.785+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02400 con 0x55af27f03400 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa725afb640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55af27f03400 0x55af27e60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa725afb640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55af27f03400 0x55af27e60100 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa729617880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa729617880 20 searching for the correct realm 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02800 con 0x55af27f03400 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa725afb640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55af27f02800 0x55af27e60c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.786+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02400 con 0x55af27f02800 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.787+0000 7fa7252fa640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55af27f02800 0x55af27e60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.787+0000 7fa7252fa640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55af27f02800 0x55af27e60c00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.787+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.787+0000 7fa7252fa640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55af27f02400 0x55af27e61180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.787+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.787+0000 7fa7262fc640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55af27f02400 0x55af27e61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55af27f02400 0x55af27e61180 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.788+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02800 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.789+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.790+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.790+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.790+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.790+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02800 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.790+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.790+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.791+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.792+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.792+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.792+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.792+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.792+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.792+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02800 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.793+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02800 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f03400 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.794+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02400 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af28975c00 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28974800 con 0x55af27f02800 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02c00 con 0x55af28975c00 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.795+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03000 con 0x55af28975c00 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03800 con 0x55af27f03400 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 zone default found 2024-04-27T05:22:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 4 Realm: () 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 zonegroup default 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03c00 con 0x55af27f02400 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.796+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960000 con 0x55af27f03400 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960400 con 0x55af27f03400 2024-04-27T05:22:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 20 started sync module instance, tier type = 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.797+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960800 con 0x55af27f02800 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.798+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.798+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94210776983808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960c00 con 0x55af27f02800 2024-04-27T05:22:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.798+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28961000 con 0x55af28975c00 2024-04-27T05:22:33.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.799+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94210776983808] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.799+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.799+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94210776984960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28961400 con 0x55af28975c00 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.799+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28961800 con 0x55af27f03400 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.800+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.800+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94210776986112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960000 con 0x55af27f03400 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.800+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960400 con 0x55af27f02800 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.800+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94210776984960] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.801+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94210776986112] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.801+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.801+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94210776987264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28961800 con 0x55af27f02800 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.801+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03800 con 0x55af28975c00 2024-04-27T05:22:33.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.802+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.802+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94210776987264] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.802+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94210776988416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f02c00 con 0x55af28975c00 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.802+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03000 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.803+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.803+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94210776989568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27f03c00 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.803+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28ad1800 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.803+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94210776988416] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.804+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(56 notify.6 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.804+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94210776990720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960800 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.804+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960c00 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.804+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94210776989568] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94210776990720] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94210776991872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960400 con 0x55af27f02400 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=0 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=1 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=2 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=3 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=4 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=5 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.805+0000 7fa729617880 20 add_watcher() i=6 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.806+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94210776991872] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.806+0000 7fa729617880 20 add_watcher() i=7 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.806+0000 7fa729617880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.806+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28961800 con 0x55af28975c00 2024-04-27T05:22:33.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.806+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.806+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94210776993024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28961000 con 0x55af28975c00 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.807+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94210776993024] v31'216 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.807+0000 7fa70cac9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.807+0000 7fa729617880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.807+0000 7fa729617880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa729617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa729617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af27ba0780 con 0x55af28961c00 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa7252fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa7252fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36094/0 (socket says 172.21.15.138:36094) 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa7252fa640 1 -- 172.21.15.138:0/1124375552 learned_addr learned my addr 172.21.15.138:0/1124375552 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa7252fa640 1 -- 172.21.15.138:0/1124375552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27bee680 con 0x55af28961c00 2024-04-27T05:22:33.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa7252fa640 1 --2- 172.21.15.138:0/1124375552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55af27ee4c60 tx=0x55af28ad8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55441a7c7f75de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.808+0000 7fa70c2c8640 1 -- 172.21.15.138:0/1124375552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af28962580 con 0x55af28961c00 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa70c2c8640 1 -- 172.21.15.138:0/1124375552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af28a4e000 con 0x55af28961c00 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa70c2c8640 1 -- 172.21.15.138:0/1124375552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af289626e0 con 0x55af28961c00 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 -- 172.21.15.138:0/1124375552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 msgr2=0x55af27e60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 --2- 172.21.15.138:0/1124375552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55af27ee4c60 tx=0x55af28ad8de0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 -- 172.21.15.138:0/1124375552 shutdown_connections 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 --2- 172.21.15.138:0/1124375552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 -- 172.21.15.138:0/1124375552 >> 172.21.15.138:0/1124375552 conn(0x55af28961400 msgr2=0x55af27c87800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 -- 172.21.15.138:0/1124375552 shutdown_connections 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 -- 172.21.15.138:0/1124375552 wait complete. 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 Processor -- start 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.809+0000 7fa729617880 1 -- start start 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa729617880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa729617880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af27ba0500 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa7252fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa7252fa640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36104/0 (socket says 172.21.15.138:36104) 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa7252fa640 1 -- 172.21.15.138:0/3658338058 learned_addr learned my addr 172.21.15.138:0/3658338058 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa7252fa640 1 -- 172.21.15.138:0/3658338058 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af27b80ea0 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa7252fa640 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55af27ee57a0 tx=0x55af28ad8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa70aac5640 1 -- 172.21.15.138:0/3658338058 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af28962840 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa70aac5640 1 -- 172.21.15.138:0/3658338058 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af28a4e780 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af27b816c0 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa70aac5640 1 -- 172.21.15.138:0/3658338058 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55af28962580 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.810+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af27b811e0 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.811+0000 7fa70aac5640 1 -- 172.21.15.138:0/3658338058 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55af27ba6b00 con 0x55af28961c00 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.811+0000 7fa70aac5640 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28960800 0x55af27e61700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.811+0000 7fa70aac5640 1 -- 172.21.15.138:0/3658338058 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55af27b8fdc0 con 0x55af28961c00 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.811+0000 7fa7262fc640 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28960800 0x55af27e61700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.811+0000 7fa729617880 5 note: GC not initialized 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.811+0000 7fa7082c0640 20 reqs_thread_entry: start 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa7262fc640 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28960800 0x55af27e61700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55af28af2c60 tx=0x55af28ad90e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa7062bc640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:04 2024 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa705a9a640 20 reqs_thread_entry: start 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff6000 con 0x55af27f02400 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff6400 con 0x55af28975c00 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.812+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff6800 con 0x55af27f02400 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~346] v31'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa729617880 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa729617880 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa729617880 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94210776995328 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff6c00 con 0x55af27f02400 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== watch-notify(notify (1) cookie 94210776991872 notify 133143986199 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55af27ecbd40 con 0x55af27f02400 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7202f0640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986199 cookie 94210776991872 notifier 4797 bl.length()=522 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7202f0640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7202f0640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7202f0640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28008000 con 0x55af27f02400 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(65 notify.7 [notify cookie 94210776995328 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== watch-notify(notify_complete (2) cookie 94210776995328 notify 133143986199 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55af27f7b860 con 0x55af27f02400 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.814+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff7000 con 0x55af28975c00 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.815+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.815+0000 7fa729617880 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.815+0000 7fa729617880 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.815+0000 7fa729617880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.815+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94210776995328 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff7400 con 0x55af27f02400 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== watch-notify(notify (1) cookie 94210776989568 notify 133143986200 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55af27f7b860 con 0x55af27f02400 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa71faef640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986200 cookie 94210776989568 notifier 4797 bl.length()=177 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa71faef640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa71faef640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa71faef640 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28960c00 con 0x55af27f02400 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(68 notify.5 [notify cookie 94210776995328 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.816+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== watch-notify(notify_complete (2) cookie 94210776995328 notify 133143986200 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55af27f7ba00 con 0x55af27f02400 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "bar", 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Bar", 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "", 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "bar", 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.838 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.839 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:33.840 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.817+0000 7fa70cac9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.817+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94210776993024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff7800 con 0x55af28975c00 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.818+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94210776993024] v31'217 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.818+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94210776983808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff7c00 con 0x55af27f02800 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.819+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94210776983808] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.819+0000 7fa729617880 20 remove_watcher() i=0 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.819+0000 7fa729617880 2 removed watcher, disabling cache 2024-04-27T05:22:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.819+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94210776984960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28008000 con 0x55af28975c00 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.820+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94210776984960] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.820+0000 7fa729617880 20 remove_watcher() i=1 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.820+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94210776986112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28008800 con 0x55af27f03400 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.821+0000 7fa725afb640 1 -- 172.21.15.138:0/1771356881 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94210776986112] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1afc0 con 0x55af27f03400 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.821+0000 7fa729617880 20 remove_watcher() i=2 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.821+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94210776987264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28008c00 con 0x55af27f02800 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.822+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94210776987264] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1b200 con 0x55af27f02800 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.822+0000 7fa729617880 20 remove_watcher() i=3 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.822+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94210776988416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28009000 con 0x55af28975c00 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.823+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94210776988416] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af28975c00 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.823+0000 7fa729617880 20 remove_watcher() i=4 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.823+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94210776989568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28009400 con 0x55af27f02400 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.824+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94210776989568] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.824+0000 7fa729617880 20 remove_watcher() i=5 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.824+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94210776990720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af28009800 con 0x55af27f02400 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.825+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94210776990720] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.825+0000 7fa729617880 20 remove_watcher() i=6 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.825+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94210776991872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55af27ff6000 con 0x55af27f02400 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa7262fc640 1 -- 172.21.15.138:0/1771356881 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94210776991872] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55af27f1a900 con 0x55af27f02400 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 20 remove_watcher() i=7 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55af27f02400 msgr2=0x55af27e61180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55af27f02400 0x55af27e61180 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55af28975c00 msgr2=0x55af27e5fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55af28975c00 0x55af27e5fb80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55af27f02800 msgr2=0x55af27e60c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55af27f02800 0x55af27e60c00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55af27f03400 msgr2=0x55af27e60100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55af27f03400 0x55af27e60100 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af27f02000 msgr2=0x55af27e5f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af27f02000 0x55af27e5f600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55af27c88c00 tx=0x55af28ad8d80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 msgr2=0x55af27e5eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/1771356881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28975800 0x55af27e5eb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55af28a399e0 tx=0x55af27c8f1a0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 reap_dead start 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa7252fa640 1 -- 172.21.15.138:0/1771356881 reap_dead start 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 shutdown_connections 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 >> 172.21.15.138:0/1771356881 conn(0x55af27b7d400 msgr2=0x55af27c87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 shutdown_connections 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/1771356881 wait complete. 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28960800 msgr2=0x55af27e61700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28960800 0x55af27e61700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55af28af2c60 tx=0x55af28ad90e0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 msgr2=0x55af27e60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.826+0000 7fa729617880 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55af27ee57a0 tx=0x55af28ad8c60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 shutdown_connections 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28960800 0x55af27e61700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 --2- 172.21.15.138:0/3658338058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af28961c00 0x55af27e60680 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 >> 172.21.15.138:0/3658338058 conn(0x55af28961400 msgr2=0x55af27c87800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 shutdown_connections 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/3658338058 wait complete. 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28ad0000 msgr2=0x55af27e5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28ad0000 0x55af27e5f080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55af28a82420 tx=0x55af27c8f4a0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 msgr2=0x55af27e5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55af28a39350 tx=0x55af27c8ec60 comp rx=0 tx=0).stop 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 shutdown_connections 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55af28ad0000 0x55af27e5f080 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 --2- 172.21.15.138:0/1063526792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af27b7d000 0x55af27e5e580 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 >> 172.21.15.138:0/1063526792 conn(0x55af27b7cc00 msgr2=0x55af27c85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 shutdown_connections 2024-04-27T05:22:33.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.827+0000 7fa729617880 1 -- 172.21.15.138:0/1063526792 wait complete. 2024-04-27T05:22:33.851 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:22:33.865 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:22:33.870 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-04-27T05:22:33.870 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-27T05:22:33.870 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-27T05:22:33.870 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-04-27T05:22:33.914 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:33.915 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa34532c880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562531609000 0x5625318ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa34532c880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5625318e0600 con 0x562531609400 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa34532c880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x562531648280 con 0x562531609000 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa341d06640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562531609000 0x5625318ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa341d06640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562531609000 0x5625318ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36120/0 (socket says 172.21.15.138:36120) 2024-04-27T05:22:33.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa341505640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x562531609400 0x562531710800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:40960/0 (socket says 172.21.15.138:40960) 2024-04-27T05:22:33.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa341d06640 1 -- 172.21.15.138:0/4054970516 learned_addr learned my addr 172.21.15.138:0/4054970516 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa341d06640 1 -- 172.21.15.138:0/4054970516 >> v1:172.21.15.114:6789/0 conn(0x562531609400 legacy=0x562531710800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.913+0000 7fa341d06640 1 -- 172.21.15.138:0/4054970516 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253160d380 con 0x562531609000 2024-04-27T05:22:33.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.915+0000 7fa341d06640 1 --2- 172.21.15.138:0/4054970516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x562531714d50 tx=0x56253171ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74f123e73351161 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.915+0000 7fa340d04640 1 -- 172.21.15.138:0/4054970516 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562531621ce0 con 0x562531609000 2024-04-27T05:22:33.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.915+0000 7fa340d04640 1 -- 172.21.15.138:0/4054970516 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625318e1b00 con 0x562531609000 2024-04-27T05:22:33.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.915+0000 7fa340d04640 1 -- 172.21.15.138:0/4054970516 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562531621e40 con 0x562531609000 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 -- 172.21.15.138:0/4054970516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 msgr2=0x5625318ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 --2- 172.21.15.138:0/4054970516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x562531714d50 tx=0x56253171ade0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 -- 172.21.15.138:0/4054970516 shutdown_connections 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 --2- 172.21.15.138:0/4054970516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 -- 172.21.15.138:0/4054970516 >> 172.21.15.138:0/4054970516 conn(0x562531608c00 msgr2=0x562531710000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 -- 172.21.15.138:0/4054970516 shutdown_connections 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 -- 172.21.15.138:0/4054970516 wait complete. 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.916+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa34532c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa34532c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562531648780 con 0x562531609000 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa341d06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa341d06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36128/0 (socket says 172.21.15.138:36128) 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa341d06640 1 -- 172.21.15.138:0/308926232 learned_addr learned my addr 172.21.15.138:0/308926232 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa341d06640 1 -- 172.21.15.138:0/308926232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253160cea0 con 0x562531609000 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa341d06640 1 --2- 172.21.15.138:0/308926232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x56253240d560 tx=0x56253171acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df12b7820f544888 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa340503640 1 -- 172.21.15.138:0/308926232 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625316214a0 con 0x562531609000 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa340503640 1 -- 172.21.15.138:0/308926232 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625318e1e00 con 0x562531609000 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.917+0000 7fa340503640 1 -- 172.21.15.138:0/308926232 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562531621ce0 con 0x562531609000 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- 172.21.15.138:0/308926232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 msgr2=0x5625318ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 --2- 172.21.15.138:0/308926232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x56253240d560 tx=0x56253171acc0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- 172.21.15.138:0/308926232 shutdown_connections 2024-04-27T05:22:33.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 --2- 172.21.15.138:0/308926232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- 172.21.15.138:0/308926232 >> 172.21.15.138:0/308926232 conn(0x562531608c00 msgr2=0x562531711000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- 172.21.15.138:0/308926232 shutdown_connections 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- 172.21.15.138:0/308926232 wait complete. 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa34532c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625316488c0 con 0x562531609000 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa341d06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa341d06640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36134/0 (socket says 172.21.15.138:36134) 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.918+0000 7fa341d06640 1 -- 172.21.15.138:0/2715299134 learned_addr learned my addr 172.21.15.138:0/2715299134 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa341d06640 1 -- 172.21.15.138:0/2715299134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253160d6c0 con 0x562531609000 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa341d06640 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x5625324c52c0 tx=0x56253171ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa33ed00640 1 -- 172.21.15.138:0/2715299134 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562531621e40 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa33ed00640 1 -- 172.21.15.138:0/2715299134 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625323e8000 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56253160d1e0 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa33ed00640 1 -- 172.21.15.138:0/2715299134 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625316214a0 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56253167aea0 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.919+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa33ed00640 1 -- 172.21.15.138:0/2715299134 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562531632b00 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa34532c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa34532c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56253162c280 con 0x562532400c00 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa33ed00640 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532401c00 0x5625318eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa33ed00640 1 -- 172.21.15.138:0/2715299134 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x56253161ba40 con 0x562531609000 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa342507640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa342507640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36150/0 (socket says 172.21.15.138:36150) 2024-04-27T05:22:33.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa342507640 1 -- 172.21.15.138:0/909438273 learned_addr learned my addr 172.21.15.138:0/909438273 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa341d06640 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532401c00 0x5625318eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa342507640 1 -- 172.21.15.138:0/909438273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253167ad00 con 0x562532400c00 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa341d06640 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532401c00 0x5625318eb080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x562532508780 tx=0x56253171bc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa342507640 1 --2- 172.21.15.138:0/909438273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x5625324c5e60 tx=0x56253171bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c08f9b9b262bc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa33c4fb640 1 -- 172.21.15.138:0/909438273 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562531621e40 con 0x562532400c00 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa33c4fb640 1 -- 172.21.15.138:0/909438273 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625323e8600 con 0x562532400c00 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.920+0000 7fa33c4fb640 1 -- 172.21.15.138:0/909438273 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625316214a0 con 0x562532400c00 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- 172.21.15.138:0/909438273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 msgr2=0x5625318eab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 --2- 172.21.15.138:0/909438273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x5625324c5e60 tx=0x56253171bf80 comp rx=0 tx=0).stop 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- 172.21.15.138:0/909438273 shutdown_connections 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 --2- 172.21.15.138:0/909438273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- 172.21.15.138:0/909438273 >> 172.21.15.138:0/909438273 conn(0x562531609400 msgr2=0x562531713000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- 172.21.15.138:0/909438273 shutdown_connections 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- 172.21.15.138:0/909438273 wait complete. 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa34532c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56253162d180 con 0x562532400c00 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa342507640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa342507640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36164/0 (socket says 172.21.15.138:36164) 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.921+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 learned_addr learned my addr 172.21.15.138:0/2816630125 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253167ab60 con 0x562532400c00 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa342507640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x5625324c53b0 tx=0x56253171bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa33acf8640 1 -- 172.21.15.138:0/2816630125 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562531621e40 con 0x562532400c00 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa33acf8640 1 -- 172.21.15.138:0/2816630125 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625323e8780 con 0x562532400c00 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa33acf8640 1 -- 172.21.15.138:0/2816630125 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625316214a0 con 0x562532400c00 2024-04-27T05:22:33.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56253167a9c0 con 0x562532400c00 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.922+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56253167a820 con 0x562532400c00 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa33acf8640 1 -- 172.21.15.138:0/2816630125 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562531633600 con 0x562532400c00 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa33acf8640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56253198a000 0x5625318eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa33acf8640 1 -- 172.21.15.138:0/2816630125 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x56253161bc00 con 0x562532400c00 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa341505640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56253198a000 0x5625318eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562532401000 0x5625318ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401400 con 0x562532401000 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa342507640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562532401000 0x5625318ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa341505640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56253198a000 0x5625318eb600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x562531715380 tx=0x562532564e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.923+0000 7fa342507640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562532401000 0x5625318ebb80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 realm 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401800 con 0x562532401000 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198b000 con 0x562532401000 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198b400 con 0x562532401000 2024-04-27T05:22:33.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56253198b800 0x5625318ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198bc00 con 0x56253198b800 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.924+0000 7fa341d06640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56253198b800 0x5625318ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa341d06640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56253198b800 0x5625318ec100 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa34532c880 20 searching for the correct realm 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198b000 con 0x56253198b800 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa341d06640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56253198b000 0x5625318ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198bc00 con 0x56253198b000 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.925+0000 7fa341505640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56253198b000 0x5625318ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.926+0000 7fa341505640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56253198b000 0x5625318ecc00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.926+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.926+0000 7fa341505640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56253198bc00 0x5625318ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.926+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.926+0000 7fa342507640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56253198bc00 0x5625318ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.926+0000 7fa342507640 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56253198bc00 0x5625318ed180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.927+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.928+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.929+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.930+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b000 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.931+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b000 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.932+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b800 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198bc00 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x562532401000 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198ac00 con 0x56253198b000 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa34532c880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:33.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa34532c880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa34532c880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa34532c880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.933+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198b400 con 0x562532401000 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401400 con 0x562532401000 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401800 con 0x56253198b800 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:33.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 zone default found 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 4 Realm: () 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 zonegroup default 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.934+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa000 con 0x56253198bc00 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa400 con 0x56253198b800 2024-04-27T05:22:33.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa800 con 0x56253198b800 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 started sync module instance, tier type = 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.935+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aac00 con 0x56253198b000 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.936+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.936+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94717745178880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab000 con 0x56253198b000 2024-04-27T05:22:33.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.936+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab400 con 0x562532401000 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.937+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94717745178880] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.937+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.937+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94717745180032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab800 con 0x562532401000 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.937+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319abc00 con 0x56253198b800 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.938+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94717745180032] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.938+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.938+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94717745181184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa400 con 0x56253198b800 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.938+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa800 con 0x56253198b000 2024-04-27T05:22:33.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.939+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94717745181184] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.939+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.939+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94717745182336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319abc00 con 0x56253198b000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.939+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa000 con 0x562532401000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.940+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94717745182336] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.941+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.941+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94717745183488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401800 con 0x562532401000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.941+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401400 con 0x56253198bc00 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.941+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.942+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94717745184640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56253198b400 con 0x56253198bc00 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.942+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532566c00 con 0x56253198bc00 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.942+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94717745183488] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.942+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94717745184640] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.943+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.943+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94717745185792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532566800 con 0x56253198bc00 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.943+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aac00 con 0x56253198bc00 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94717745185792] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94717745186944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab000 con 0x56253198bc00 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=0 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=1 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=2 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=3 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=4 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=5 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=6 2024-04-27T05:22:33.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94717745186944] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 20 add_watcher() i=7 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.944+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa800 con 0x562532401000 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.945+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.945+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94717745188096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab400 con 0x562532401000 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94717745188096] v31'218 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa328cd4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa34532c880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa34532c880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa34532c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.946+0000 7fa34532c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56253162c780 con 0x56253198ac00 2024-04-27T05:22:33.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa341505640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa341505640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36170/0 (socket says 172.21.15.138:36170) 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa341505640 1 -- 172.21.15.138:0/3277891568 learned_addr learned my addr 172.21.15.138:0/3277891568 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa341505640 1 -- 172.21.15.138:0/3277891568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253167a680 con 0x56253198ac00 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa341505640 1 --2- 172.21.15.138:0/3277891568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x562531968ae0 tx=0x562532564de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5773381cbd066faf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa3284d3640 1 -- 172.21.15.138:0/3277891568 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625319ac580 con 0x56253198ac00 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa3284d3640 1 -- 172.21.15.138:0/3277891568 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625323e8000 con 0x56253198ac00 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.947+0000 7fa3284d3640 1 -- 172.21.15.138:0/3277891568 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625319ac6e0 con 0x56253198ac00 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- 172.21.15.138:0/3277891568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 msgr2=0x5625318ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 --2- 172.21.15.138:0/3277891568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x562531968ae0 tx=0x562532564de0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- 172.21.15.138:0/3277891568 shutdown_connections 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 --2- 172.21.15.138:0/3277891568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- 172.21.15.138:0/3277891568 >> 172.21.15.138:0/3277891568 conn(0x5625319ab800 msgr2=0x562531713800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- 172.21.15.138:0/3277891568 shutdown_connections 2024-04-27T05:22:33.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- 172.21.15.138:0/3277891568 wait complete. 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 Processor -- start 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- start start 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa34532c880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56253162c500 con 0x56253198ac00 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa341505640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa341505640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:36176/0 (socket says 172.21.15.138:36176) 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.948+0000 7fa341505640 1 -- 172.21.15.138:0/626547267 learned_addr learned my addr 172.21.15.138:0/626547267 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa341505640 1 -- 172.21.15.138:0/626547267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56253167a4e0 con 0x56253198ac00 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa341505640 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x562531969830 tx=0x562532565560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa326cd0640 1 -- 172.21.15.138:0/626547267 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625319ac840 con 0x56253198ac00 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa326cd0640 1 -- 172.21.15.138:0/626547267 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625323e8a80 con 0x56253198ac00 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56253160d380 con 0x56253198ac00 2024-04-27T05:22:33.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa326cd0640 1 -- 172.21.15.138:0/626547267 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625319ac580 con 0x56253198ac00 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.949+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56253160cea0 con 0x56253198ac00 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa326cd0640 1 -- 172.21.15.138:0/626547267 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562531633b80 con 0x56253198ac00 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa326cd0640 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532566c00 0x5625318ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa326cd0640 1 -- 172.21.15.138:0/626547267 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x56253161bdc0 con 0x56253198ac00 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa34532c880 5 note: GC not initialized 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa342507640 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532566c00 0x5625318ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa3244cb640 20 reqs_thread_entry: start 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa34532c880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa342507640 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532566c00 0x5625318ed700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5625325721b0 tx=0x562532565320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa34532c880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa3224c7640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:04 2024 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:33.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa321ca5640 20 reqs_thread_entry: start 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa34532c880 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.950+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e000 con 0x56253198bc00 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.951+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 bar [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.951+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.951+0000 7fa34532c880 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.951+0000 7fa34532c880 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.951+0000 7fa34532c880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.951+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e400 con 0x56253198b000 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.952+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 bud [create,call,writefull 0~181] v31'4 uv4 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.952+0000 7fa34532c880 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.952+0000 7fa34532c880 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.952+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T05:22:33.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.952+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 94717745190400 in=369b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e800 con 0x56253198bc00 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== watch-notify(notify (1) cookie 94717745184640 notify 133143986202 ret 0) v3 ==== 399+0+0 (crc 0 0 0) 0x56253194dd40 con 0x56253198bc00 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa33c4fb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986202 cookie 94717745184640 notifier 4805 bl.length()=357 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(64 notify.5 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa33c4fb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa33c4fb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa33c4fb640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92000 con 0x56253198bc00 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986202 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562531a05a00 con 0x56253198bc00 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7ec00 con 0x562532401000 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv4 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa34532c880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 : miss 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.953+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 8.5 8:a5333f3b:root::.bucket.meta.myfoo6%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7f000 con 0x562532401000 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(67 .bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=353 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x17 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.954+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 8.5 8:a5333f3b:root::.bucket.meta.myfoo6%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7f800 con 0x562532401000 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(68 .bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v31'5 uv5 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa34532c880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x17 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa34532c880 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa34532c880 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T05:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7, ofs0, ns] 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify cookie 94717745190400 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92000 con 0x56253198bc00 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.955+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== watch-notify(notify (1) cookie 94717745184640 notify 133143986203 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x562531a05a00 con 0x56253198bc00 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa33c4fb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986203 cookie 94717745184640 notifier 4805 bl.length()=721 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(69 notify.5 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa33c4fb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x17 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa33c4fb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa33c4fb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa33c4fb640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92400 con 0x56253198bc00 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(70 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986203 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562531a05ba0 con 0x56253198bc00 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa34532c880 20 handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): orig_sources= new_sources= 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa34532c880 20 handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): potential sources added=[] removed=[] 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa34532c880 20 handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): orig_dests= new_dests= 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa34532c880 20 handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): potential dests added=[] removed=[] 2024-04-27T05:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.956+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92800 con 0x562532401000 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(71 bar.buckets [call] v31'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 20 rados->read ofs=0 len=0 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92c00 con 0x562532401000 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(72 myfoo6 [call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv16 ondisk = 0) v8 ==== 234+0+289 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 20 rados_obj.operate() r=0 bl.length=166 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x13 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.957+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93400 con 0x562532401000 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.958+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(73 myfoo6 [delete,create,call,call,writefull 0~166] v31'17 uv17 ondisk = 0) v8 ==== 318+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.958+0000 7fa34532c880 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.958+0000 7fa34532c880 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.958+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T05:22:33.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 94717745190400 in=352b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93800 con 0x56253198b000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== watch-notify(notify (1) cookie 94717745182336 notify 133143986191 ret 0) v3 ==== 382+0+0 (crc 0 0 0) 0x562531a05d40 con 0x56253198b000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa33c4fb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 94717745182336 notifier 4805 bl.length()=340 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa33c4fb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa33c4fb640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(74 notify.3 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa33c4fb640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7fc00 con 0x56253198b000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986191 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562531a01860 con 0x56253198b000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.959+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:76 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93c00 con 0x562532401000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.960+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(76 bud.buckets [call] v31'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.960+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e000 con 0x562532401000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(77 myfoo6 [delete,create,call,writefull 0~166] v31'18 uv18 ondisk = 0) v8 ==== 276+0+0 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa34532c880 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa34532c880 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T05:22:33.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [notify cookie 94717745190400 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e800 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== watch-notify(notify (1) cookie 94717745182336 notify 133143986192 ret 0) v3 ==== 358+0+0 (crc 0 0 0) 0x562531a01860 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa33bcfa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94717745182336 notifier 4805 bl.length()=316 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(78 notify.3 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa33bcfa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa33bcfa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.961+0000 7fa33bcfa640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:79 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aa000 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.962+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986192 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5625319feea0 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.962+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(79 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.962+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 4.1a 4:5aa84941:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7ec00 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.962+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(80 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.962+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 4.23 4:c42703a1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7f000 con 0x56253198bc00 2024-04-27T05:22:33.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.962+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(81 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199a900 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 4.38 4:1d132ca1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aac00 con 0x562532401000 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(82 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199a900 con 0x562532401000 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 4.1e 4:78bf5de1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab000 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(83 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 4.1c 4:395ac92c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93800 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(84 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.963+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 4.1c 4:3a8f09fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92000 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.964+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(85 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.964+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:86 4.19 4:9aea267d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92400 con 0x56253198b800 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.964+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(86 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.964+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:87 4.2b 4:d4d5e5db:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92800 con 0x562532401000 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.964+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(87 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.964+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:88 4.6 4:630ee69a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92c00 con 0x56253198b800 2024-04-27T05:22:33.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(88 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93400 con 0x562532401000 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(89 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [call out=378b] v0'0 uv11 ondisk = 0) v8 ==== 194+0+378 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa34532c880 10 cls_bucket_list_unordered: got thirteen 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:90 4.11 4:89256115:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e400 con 0x56253198bc00 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== osd_op_reply(90 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa34532c880 20 get_obj_state: rctx=0x562531a5e2a0 obj=myfoo6:thirteen state=0x56253242cde8 s->prefetch_data=0 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.965+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:91 3.9 3:90a65acf:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7fc00 con 0x56253198b800 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(91 2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen [getxattrs out=856b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+872 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 10 manifest: total_size = 8 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.17747013688768783099 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.acl 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.content_type 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.etag 2024-04-27T05:22:33.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.idtag 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.manifest 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.source_zone 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 get_obj_state: rctx=0x562531a5e2a0 obj=myfoo6:thirteen state=0x56253242cde8 s->prefetch_data=0 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 : hit (requested=0x17, cached=0x17) 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.966+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:92 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e800 con 0x562532401000 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.967+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(92 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [stat,call,call] v31'12 uv12 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.967+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:93 3.9 3:90a65acf:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen:head [cmpxattr user.rgw.idtag (63) op 1 mode 1 in=77b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562532401400 con 0x56253198b800 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.968+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(93 2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen [cmpxattr (63) op 1 mode 1,setxattr (117),setxattr (33)] v31'2 uv2 ondisk = 0) v8 ==== 280+0+0 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.968+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:94 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93c00 con 0x562532401000 2024-04-27T05:22:33.993 INFO:teuthology.orchestra.run.smithi138.stderr:1 objects processed in :myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]). Next marker thirteen 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.968+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93800 con 0x56253198b000 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(94 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [stat,call,call] v31'13 uv13 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa33bcfa640 20 handle_completion(): completion ok for obj=thirteen 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(95 bud [delete,create,call,writefull 0~346] v31'5 uv5 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa34532c880 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa34532c880 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify cookie 94717745190400 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92000 con 0x56253198bc00 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== watch-notify(notify (1) cookie 94717745184640 notify 133143986204 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x562531a05ba0 con 0x56253198bc00 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa33c4fb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986204 cookie 94717745184640 notifier 4805 bl.length()=522 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa33c4fb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa33c4fb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:22:33.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== osd_op_reply(96 notify.5 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa33c4fb640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7f400 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 33 ==== osd_op_reply(97 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 34 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986204 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562531a041a0 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.969+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:98 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92800 con 0x562532401000 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(98 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v31'2 uv2 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa34532c880 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa34532c880 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify cookie 94717745190400 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aac00 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 35 ==== watch-notify(notify (1) cookie 94717745184640 notify 133143986205 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x562531a041a0 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa33bcfa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986205 cookie 94717745184640 notifier 4805 bl.length()=177 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa33bcfa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa33bcfa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 36 ==== osd_op_reply(99 notify.5 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa33bcfa640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93000 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 37 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986205 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562531a04340 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa34532c880 10 removing user index: bar 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 38 ==== osd_op_reply(100 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.971+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:101 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab000 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 39 ==== osd_op_reply(101 bar [delete] v31'4 uv4 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa34532c880 10 removing default.rgw.meta+users.uid+bar from cache 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa34532c880 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify cookie 94717745190400 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e000 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 40 ==== watch-notify(notify (1) cookie 94717745186944 notify 133143986206 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x562531a04340 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa33c4fb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986206 cookie 94717745186944 notifier 4805 bl.length()=152 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 41 ==== osd_op_reply(102 notify.7 [notify cookie 94717745190400 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.972+0000 7fa33c4fb640 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7f800 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.973+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 42 ==== osd_op_reply(103 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:33.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.973+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 43 ==== watch-notify(notify_complete (2) cookie 94717745190400 notify 133143986206 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562531a044e0 con 0x56253198bc00 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "bud", 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Bar", 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "", 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:22:33.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "bud", 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:22:33.998 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:22:33.999 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:22:34.000 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:22:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:34.000 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:22:34.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.973+0000 7fa328cd4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:34.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.973+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:104 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94717745188096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7f000 con 0x562532401000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.974+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(104 data_loggenerations_metadata [watch unwatch cookie 94717745188096] v31'219 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.974+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94717745178880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319aac00 con 0x56253198b000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.975+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94717745178880] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.975+0000 7fa34532c880 20 remove_watcher() i=0 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.975+0000 7fa34532c880 2 removed watcher, disabling cache 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.975+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94717745180032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625319ab000 con 0x562532401000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.976+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 36 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94717745180032] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.976+0000 7fa34532c880 20 remove_watcher() i=1 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.976+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94717745181184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93400 con 0x56253198b800 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.977+0000 7fa341d06640 1 -- 172.21.15.138:0/2816630125 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94717745181184] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199afc0 con 0x56253198b800 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.977+0000 7fa34532c880 20 remove_watcher() i=2 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.977+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94717745182336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93c00 con 0x56253198b000 2024-04-27T05:22:34.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.978+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94717745182336] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199b200 con 0x56253198b000 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.978+0000 7fa34532c880 20 remove_watcher() i=3 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.978+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94717745183488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92000 con 0x562532401000 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.979+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.1 v2:172.21.15.138:6800/29864 37 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94717745183488] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x562532401000 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.979+0000 7fa34532c880 20 remove_watcher() i=4 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.979+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94717745184640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a92800 con 0x56253198bc00 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.980+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 44 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94717745184640] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.980+0000 7fa34532c880 20 remove_watcher() i=5 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.980+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94717745185792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a93000 con 0x56253198bc00 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.981+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 45 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94717745185792] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.981+0000 7fa34532c880 20 remove_watcher() i=6 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.981+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94717745186944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562531a7e400 con 0x56253198bc00 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa342507640 1 -- 172.21.15.138:0/2816630125 <== osd.0 v2:172.21.15.114:6800/27747 46 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94717745186944] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56253199ab40 con 0x56253198bc00 2024-04-27T05:22:34.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 20 remove_watcher() i=7 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56253198bc00 msgr2=0x5625318ed180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56253198bc00 0x5625318ed180 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562532401000 msgr2=0x5625318ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562532401000 0x5625318ebb80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56253198b000 msgr2=0x5625318ecc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56253198b000 0x5625318ecc00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56253198b800 msgr2=0x5625318ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56253198b800 0x5625318ec100 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56253198a000 msgr2=0x5625318eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56253198a000 0x5625318eb600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x562531715380 tx=0x562532564e40 comp rx=0 tx=0).stop 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 msgr2=0x5625318eab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 --2- 172.21.15.138:0/2816630125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562532400c00 0x5625318eab00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x5625324c53b0 tx=0x56253171bb00 comp rx=0 tx=0).stop 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 reap_dead start 2024-04-27T05:22:34.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa341505640 1 -- 172.21.15.138:0/2816630125 reap_dead start 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.982+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 shutdown_connections 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 >> 172.21.15.138:0/2816630125 conn(0x562531609400 msgr2=0x562531713000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 shutdown_connections 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2816630125 wait complete. 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532566c00 msgr2=0x5625318ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532566c00 0x5625318ed700 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5625325721b0 tx=0x562532565320 comp rx=0 tx=0).stop 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 msgr2=0x5625318ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x562531969830 tx=0x562532565560 comp rx=0 tx=0).stop 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 shutdown_connections 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532566c00 0x5625318ed700 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/626547267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56253198ac00 0x5625318ec680 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 >> 172.21.15.138:0/626547267 conn(0x5625319ab800 msgr2=0x562531713800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 shutdown_connections 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/626547267 wait complete. 2024-04-27T05:22:34.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532401c00 msgr2=0x5625318eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532401c00 0x5625318eb080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x562532508780 tx=0x56253171bc20 comp rx=0 tx=0).stop 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 msgr2=0x5625318ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x5625324c52c0 tx=0x56253171ac60 comp rx=0 tx=0).stop 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 shutdown_connections 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562532401c00 0x5625318eb080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 --2- 172.21.15.138:0/2715299134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562531609000 0x5625318ea580 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 >> 172.21.15.138:0/2715299134 conn(0x562531608c00 msgr2=0x562531711000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.983+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 shutdown_connections 2024-04-27T05:22:34.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:33.984+0000 7fa34532c880 1 -- 172.21.15.138:0/2715299134 wait complete. 2024-04-27T05:22:34.010 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:22:39.017 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:22:39.017 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-27T05:22:39.017 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-27T05:22:39.017 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-27T05:22:39.018 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-04-27T05:22:39.074 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.074+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.074+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.074+0000 7fc8a4528880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.074+0000 7fc8a4528880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564f4b480600 con 0x564f4b1a9400 2024-04-27T05:22:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.074+0000 7fc8a4528880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x564f4b1e8280 con 0x564f4b1a9000 2024-04-27T05:22:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.075+0000 7fc8a0e4d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.075+0000 7fc8a064c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x564f4b1a9400 0x564f4b2b0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:48274/0 (socket says 172.21.15.138:48274) 2024-04-27T05:22:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.075+0000 7fc8a064c640 1 -- 172.21.15.138:0/1285426494 learned_addr learned my addr 172.21.15.138:0/1285426494 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.075+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/1285426494 >> v1:172.21.15.114:6789/0 conn(0x564f4b1a9400 legacy=0x564f4b2b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.075+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/1285426494 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b1ad380 con 0x564f4b1a9000 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/1285426494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x564f4b2b4d50 tx=0x564f4b2bade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f96d02f4e22eb79 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc89fe4b640 1 -- 172.21.15.138:0/1285426494 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c1ce0 con 0x564f4b1a9000 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc89fe4b640 1 -- 172.21.15.138:0/1285426494 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4b481b00 con 0x564f4b1a9000 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc89fe4b640 1 -- 172.21.15.138:0/1285426494 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c1e40 con 0x564f4b1a9000 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 -- 172.21.15.138:0/1285426494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 msgr2=0x564f4b48a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 --2- 172.21.15.138:0/1285426494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x564f4b2b4d50 tx=0x564f4b2bade0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 -- 172.21.15.138:0/1285426494 shutdown_connections 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 --2- 172.21.15.138:0/1285426494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 -- 172.21.15.138:0/1285426494 >> 172.21.15.138:0/1285426494 conn(0x564f4b1a8c00 msgr2=0x564f4b2b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 -- 172.21.15.138:0/1285426494 shutdown_connections 2024-04-27T05:22:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.077+0000 7fc8a4528880 1 -- 172.21.15.138:0/1285426494 wait complete. 2024-04-27T05:22:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.078+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.078+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.078+0000 7fc8a4528880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.078+0000 7fc8a4528880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4b1e8780 con 0x564f4b1a9000 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc8a0e4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc8a0e4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52292/0 (socket says 172.21.15.138:52292) 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/3389647106 learned_addr learned my addr 172.21.15.138:0/3389647106 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/3389647106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b1acea0 con 0x564f4b1a9000 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/3389647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x564f4bfad560 tx=0x564f4b2bacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ea41bf835dbbec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc89f64a640 1 -- 172.21.15.138:0/3389647106 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c14a0 con 0x564f4b1a9000 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc89f64a640 1 -- 172.21.15.138:0/3389647106 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4b481e00 con 0x564f4b1a9000 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.079+0000 7fc89f64a640 1 -- 172.21.15.138:0/3389647106 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c1ce0 con 0x564f4b1a9000 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- 172.21.15.138:0/3389647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 msgr2=0x564f4b48a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 --2- 172.21.15.138:0/3389647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x564f4bfad560 tx=0x564f4b2bacc0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- 172.21.15.138:0/3389647106 shutdown_connections 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 --2- 172.21.15.138:0/3389647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- 172.21.15.138:0/3389647106 >> 172.21.15.138:0/3389647106 conn(0x564f4b1a8c00 msgr2=0x564f4b2b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- 172.21.15.138:0/3389647106 shutdown_connections 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- 172.21.15.138:0/3389647106 wait complete. 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a4528880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4b1e88c0 con 0x564f4b1a9000 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.080+0000 7fc8a0e4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc8a0e4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52300/0 (socket says 172.21.15.138:52300) 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/236704977 learned_addr learned my addr 172.21.15.138:0/236704977 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/236704977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b1ad6c0 con 0x564f4b1a9000 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x564f4c064de0 tx=0x564f4b2bac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc89de47640 1 -- 172.21.15.138:0/236704977 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c1e40 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc89de47640 1 -- 172.21.15.138:0/236704977 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4bf88000 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc89de47640 1 -- 172.21.15.138:0/236704977 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c14a0 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f4b1ad1e0 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.081+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f4b21aea0 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a4528880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc89de47640 1 -- 172.21.15.138:0/236704977 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564f4b1d2b00 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a4528880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4b1cc280 con 0x564f4bfa0c00 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc89de47640 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4bfa1c00 0x564f4b48b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc89de47640 1 -- 172.21.15.138:0/236704977 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x564f4b1bba40 con 0x564f4b1a9000 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a164e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a164e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52312/0 (socket says 172.21.15.138:52312) 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a164e640 1 -- 172.21.15.138:0/3984011038 learned_addr learned my addr 172.21.15.138:0/3984011038 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.082+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4bfa1c00 0x564f4b48b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a164e640 1 -- 172.21.15.138:0/3984011038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b21ad00 con 0x564f4bfa0c00 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4bfa1c00 0x564f4b48b080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564f4c0b22a0 tx=0x564f4b2bbc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a164e640 1 --2- 172.21.15.138:0/3984011038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x564f4c065620 tx=0x564f4b2bbf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31e92becbf09b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc89b642640 1 -- 172.21.15.138:0/3984011038 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c1e40 con 0x564f4bfa0c00 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc89b642640 1 -- 172.21.15.138:0/3984011038 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4bf88600 con 0x564f4bfa0c00 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc89b642640 1 -- 172.21.15.138:0/3984011038 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c14a0 con 0x564f4bfa0c00 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 -- 172.21.15.138:0/3984011038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 msgr2=0x564f4b48ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 --2- 172.21.15.138:0/3984011038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x564f4c065620 tx=0x564f4b2bbf80 comp rx=0 tx=0).stop 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 -- 172.21.15.138:0/3984011038 shutdown_connections 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 --2- 172.21.15.138:0/3984011038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 -- 172.21.15.138:0/3984011038 >> 172.21.15.138:0/3984011038 conn(0x564f4b1a9400 msgr2=0x564f4b2b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 -- 172.21.15.138:0/3984011038 shutdown_connections 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.083+0000 7fc8a4528880 1 -- 172.21.15.138:0/3984011038 wait complete. 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a4528880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a4528880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4b1cd180 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a164e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a164e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52326/0 (socket says 172.21.15.138:52326) 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 learned_addr learned my addr 172.21.15.138:0/4239427965 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.084+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b21ab60 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.085+0000 7fc8a164e640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x564f4bfad5c0 tx=0x564f4b2bbb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.085+0000 7fc899e3f640 1 -- 172.21.15.138:0/4239427965 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c1e40 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.085+0000 7fc899e3f640 1 -- 172.21.15.138:0/4239427965 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4bf88780 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.085+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f4b21a9c0 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.085+0000 7fc899e3f640 1 -- 172.21.15.138:0/4239427965 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b1c14a0 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.085+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f4b21a820 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc899e3f640 1 -- 172.21.15.138:0/4239427965 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564f4b1d3600 con 0x564f4bfa0c00 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc899e3f640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4b530000 0x564f4b48b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc899e3f640 1 -- 172.21.15.138:0/4239427965 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x564f4b1bbc00 con 0x564f4bfa0c00 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564f4bfa1000 0x564f4b48bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a064c640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4b530000 0x564f4b48b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4bfa1400 con 0x564f4bfa1000 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a164e640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564f4bfa1000 0x564f4b48bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a064c640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4b530000 0x564f4b48b600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x564f4c11cf60 tx=0x564f4c10ef60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.086+0000 7fc8a164e640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564f4bfa1000 0x564f4b48bb80 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 realm 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4bfa1800 con 0x564f4bfa1000 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531000 con 0x564f4bfa1000 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531400 con 0x564f4bfa1000 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x564f4b531800 0x564f4b48c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.087+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531c00 con 0x564f4b531800 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x564f4b531800 0x564f4b48c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x564f4b531800 0x564f4b48c100 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a4528880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a4528880 20 searching for the correct realm 2024-04-27T05:22:39.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531000 con 0x564f4b531800 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a0e4d640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x564f4b531000 0x564f4b48cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.088+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531c00 con 0x564f4b531000 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.089+0000 7fc8a064c640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x564f4b531000 0x564f4b48cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.089+0000 7fc8a064c640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x564f4b531000 0x564f4b48cc00 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.089+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.089+0000 7fc8a064c640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x564f4b531c00 0x564f4b48d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.089+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.089+0000 7fc8a164e640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x564f4b531c00 0x564f4b48d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.090+0000 7fc8a164e640 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x564f4b531c00 0x564f4b48d180 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.090+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.090+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.090+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.090+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531000 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.091+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.092+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531000 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.093+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.094+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.095+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.096+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.096+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.096+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.096+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.096+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.096+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531000 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531800 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531c00 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4bfa1000 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.097+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b530c00 con 0x564f4b531000 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531400 con 0x564f4bfa1000 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4bfa1400 con 0x564f4bfa1000 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4bfa1800 con 0x564f4b531800 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 zone default found 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 4 Realm: () 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 zonegroup default 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.098+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556000 con 0x564f4b531c00 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556400 con 0x564f4b531800 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556800 con 0x564f4b531800 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 started sync module instance, tier type = 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.099+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556c00 con 0x564f4b531000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.100+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.100+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94898565425408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557000 con 0x564f4b531000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.100+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557400 con 0x564f4bfa1000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.101+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94898565425408] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.102+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.102+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94898565426560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557800 con 0x564f4bfa1000 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.102+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557c00 con 0x564f4b531800 2024-04-27T05:22:39.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.103+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.103+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94898565427712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556400 con 0x564f4b531800 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.103+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556800 con 0x564f4b531000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.103+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94898565426560] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.104+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94898565427712] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.104+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.104+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94898565428864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557c00 con 0x564f4b531000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.104+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556000 con 0x564f4bfa1000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.105+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.105+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94898565428864] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.105+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94898565430016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4bfa1800 con 0x564f4bfa1000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.105+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4bfa1400 con 0x564f4b531c00 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.106+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94898565430016] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.106+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.106+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94898565431168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b531400 con 0x564f4b531c00 2024-04-27T05:22:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.106+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4c110c00 con 0x564f4b531c00 2024-04-27T05:22:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.107+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94898565431168] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.107+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.108+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94898565432320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4c110800 con 0x564f4b531c00 2024-04-27T05:22:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.108+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556c00 con 0x564f4b531c00 2024-04-27T05:22:39.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94898565432320] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94898565433472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557000 con 0x564f4b531c00 2024-04-27T05:22:39.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=0 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=1 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=2 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=3 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=4 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=5 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.109+0000 7fc8a4528880 20 add_watcher() i=6 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.110+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94898565433472] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.110+0000 7fc8a4528880 20 add_watcher() i=7 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.110+0000 7fc8a4528880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.110+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b556800 con 0x564f4bfa1000 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.110+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.110+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94898565434624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b557400 con 0x564f4bfa1000 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.111+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94898565434624] v31'220 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.111+0000 7fc887e1b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.111+0000 7fc8a4528880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.111+0000 7fc8a4528880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.111+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a4528880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a4528880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4b1cc780 con 0x564f4b530c00 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a064c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a064c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52340/0 (socket says 172.21.15.138:52340) 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a064c640 1 -- 172.21.15.138:0/701741592 learned_addr learned my addr 172.21.15.138:0/701741592 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a064c640 1 -- 172.21.15.138:0/701741592 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b21a680 con 0x564f4b530c00 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc8a064c640 1 --2- 172.21.15.138:0/701741592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x564f4b512960 tx=0x564f4c10ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766efb4306a9ac18 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.112+0000 7fc88761a640 1 -- 172.21.15.138:0/701741592 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b558580 con 0x564f4b530c00 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc88761a640 1 -- 172.21.15.138:0/701741592 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4bf88000 con 0x564f4b530c00 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc88761a640 1 -- 172.21.15.138:0/701741592 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b5586e0 con 0x564f4b530c00 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 -- 172.21.15.138:0/701741592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 msgr2=0x564f4b48c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 --2- 172.21.15.138:0/701741592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x564f4b512960 tx=0x564f4c10ef00 comp rx=0 tx=0).stop 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 -- 172.21.15.138:0/701741592 shutdown_connections 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 --2- 172.21.15.138:0/701741592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 -- 172.21.15.138:0/701741592 >> 172.21.15.138:0/701741592 conn(0x564f4b557800 msgr2=0x564f4b2b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 -- 172.21.15.138:0/701741592 shutdown_connections 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.113+0000 7fc8a4528880 1 -- 172.21.15.138:0/701741592 wait complete. 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a4528880 1 Processor -- start 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a4528880 1 -- start start 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a4528880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a4528880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4b1cc500 con 0x564f4b530c00 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a064c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a064c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52354/0 (socket says 172.21.15.138:52354) 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a064c640 1 -- 172.21.15.138:0/2597606768 learned_addr learned my addr 172.21.15.138:0/2597606768 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a064c640 1 -- 172.21.15.138:0/2597606768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f4b21a4e0 con 0x564f4b530c00 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc8a064c640 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x564f4b513890 tx=0x564f4c10f080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.114+0000 7fc885e17640 1 -- 172.21.15.138:0/2597606768 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b558840 con 0x564f4b530c00 2024-04-27T05:22:39.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc885e17640 1 -- 172.21.15.138:0/2597606768 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f4bf88a80 con 0x564f4b530c00 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f4b1ad380 con 0x564f4b530c00 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc885e17640 1 -- 172.21.15.138:0/2597606768 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f4b558580 con 0x564f4b530c00 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f4b1acea0 con 0x564f4b530c00 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc885e17640 1 -- 172.21.15.138:0/2597606768 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x564f4b1d3b80 con 0x564f4b530c00 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc885e17640 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4c110c00 0x564f4b48d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.115+0000 7fc885e17640 1 -- 172.21.15.138:0/2597606768 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x564f4b1bbdc0 con 0x564f4b530c00 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a4528880 5 note: GC not initialized 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a164e640 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4c110c00 0x564f4b48d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc883612640 20 reqs_thread_entry: start 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a4528880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a164e640 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4c110c00 0x564f4b48d700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x564f4c11c6f0 tx=0x564f4c10fe60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a4528880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc88160e640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:09 2024 2024-04-27T05:22:39.130 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc880dec640 20 reqs_thread_entry: start 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a4528880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b628000 con 0x564f4b531c00 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.116+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b628400 con 0x564f4b531c00 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T05:22:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.117+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b628800 con 0x564f4b531c00 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v31'5 uv5 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc8a4528880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc8a4528880 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc8a4528880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94898565436928 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b628c00 con 0x564f4b531c00 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== watch-notify(notify (1) cookie 94898565431168 notify 133143986207 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x564f4b4f7d40 con 0x564f4b531c00 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc89b642640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986207 cookie 94898565431168 notifier 4813 bl.length()=522 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc89b642640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.118+0000 7fc89b642640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.119+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(65 notify.5 [notify cookie 94898565436928 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.119+0000 7fc89b642640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b638000 con 0x564f4b531c00 2024-04-27T05:22:39.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.119+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== watch-notify(notify_complete (2) cookie 94898565436928 notify 133143986207 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x564f4b5afa00 con 0x564f4b531c00 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.119+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.119+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b629000 con 0x564f4b531c00 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'9 uv9 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a4528880 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a4528880 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a4528880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94898565436928 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b629400 con 0x564f4b531800 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94898565427712 notify 133143986195 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x564f4c0b8d00 con 0x564f4b531800 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc89b642640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986195 cookie 94898565427712 notifier 4813 bl.length()=177 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc89b642640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc89b642640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [notify cookie 94898565436928 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.120+0000 7fc89b642640 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b638400 con 0x564f4b531800 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.121+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== watch-notify(notify_complete (2) cookie 94898565436928 notify 133143986195 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x564f4c0b8ea0 con 0x564f4b531800 2024-04-27T05:22:39.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.121+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "fud", 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Fud", 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "", 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 4, 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "fud", 2024-04-27T05:22:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [], 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:22:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:22:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:22:39.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.121+0000 7fc887e1b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.121+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94898565434624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b629800 con 0x564f4bfa1000 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.123+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94898565434624] v31'221 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.123+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94898565425408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b629c00 con 0x564f4b531000 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.124+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94898565425408] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.124+0000 7fc8a4528880 20 remove_watcher() i=0 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.124+0000 7fc8a4528880 2 removed watcher, disabling cache 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.124+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94898565426560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b638000 con 0x564f4bfa1000 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.125+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94898565426560] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.125+0000 7fc8a4528880 20 remove_watcher() i=1 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.125+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94898565427712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b638800 con 0x564f4b531800 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.126+0000 7fc8a0e4d640 1 -- 172.21.15.138:0/4239427965 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94898565427712] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548fc0 con 0x564f4b531800 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.126+0000 7fc8a4528880 20 remove_watcher() i=2 2024-04-27T05:22:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.126+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94898565428864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b638c00 con 0x564f4b531000 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.127+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94898565428864] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b549200 con 0x564f4b531000 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.127+0000 7fc8a4528880 20 remove_watcher() i=3 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.127+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94898565430016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b639000 con 0x564f4bfa1000 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.128+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94898565430016] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4bfa1000 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.128+0000 7fc8a4528880 20 remove_watcher() i=4 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.128+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94898565431168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b639400 con 0x564f4b531c00 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.129+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94898565431168] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.129+0000 7fc8a4528880 20 remove_watcher() i=5 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.129+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94898565432320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b639800 con 0x564f4b531c00 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.130+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94898565432320] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.130+0000 7fc8a4528880 20 remove_watcher() i=6 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.130+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94898565433472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x564f4b639c00 con 0x564f4b531c00 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a164e640 1 -- 172.21.15.138:0/4239427965 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94898565433472] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f4b548900 con 0x564f4b531c00 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 20 remove_watcher() i=7 2024-04-27T05:22:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x564f4b531c00 msgr2=0x564f4b48d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x564f4b531c00 0x564f4b48d180 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564f4bfa1000 msgr2=0x564f4b48bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x564f4bfa1000 0x564f4b48bb80 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x564f4b531000 msgr2=0x564f4b48cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x564f4b531000 0x564f4b48cc00 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x564f4b531800 msgr2=0x564f4b48c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x564f4b531800 0x564f4b48c100 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4b530000 msgr2=0x564f4b48b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4b530000 0x564f4b48b600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x564f4c11cf60 tx=0x564f4c10ef60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 msgr2=0x564f4b48ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 --2- 172.21.15.138:0/4239427965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4bfa0c00 0x564f4b48ab00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x564f4bfad5c0 tx=0x564f4b2bbb00 comp rx=0 tx=0).stop 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 reap_dead start 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a064c640 1 -- 172.21.15.138:0/4239427965 reap_dead start 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 shutdown_connections 2024-04-27T05:22:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.131+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 >> 172.21.15.138:0/4239427965 conn(0x564f4b1a9400 msgr2=0x564f4b2b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 shutdown_connections 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/4239427965 wait complete. 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4c110c00 msgr2=0x564f4b48d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4c110c00 0x564f4b48d700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x564f4c11c6f0 tx=0x564f4c10fe60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 msgr2=0x564f4b48c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x564f4b513890 tx=0x564f4c10f080 comp rx=0 tx=0).stop 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 shutdown_connections 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4c110c00 0x564f4b48d700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/2597606768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b530c00 0x564f4b48c680 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 >> 172.21.15.138:0/2597606768 conn(0x564f4b557800 msgr2=0x564f4b2b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 shutdown_connections 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/2597606768 wait complete. 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4bfa1c00 msgr2=0x564f4b48b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4bfa1c00 0x564f4b48b080 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564f4c0b22a0 tx=0x564f4b2bbc20 comp rx=0 tx=0).stop 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 msgr2=0x564f4b48a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x564f4c064de0 tx=0x564f4b2bac60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 shutdown_connections 2024-04-27T05:22:39.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x564f4bfa1c00 0x564f4b48b080 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 --2- 172.21.15.138:0/236704977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564f4b1a9000 0x564f4b48a580 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.132+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 >> 172.21.15.138:0/236704977 conn(0x564f4b1a8c00 msgr2=0x564f4b2b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.133+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 shutdown_connections 2024-04-27T05:22:39.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.133+0000 7fc8a4528880 1 -- 172.21.15.138:0/236704977 wait complete. 2024-04-27T05:22:39.146 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:22:39.158 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:22:39.162 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-04-27T05:22:39.162 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-27T05:22:39.163 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-27T05:22:39.163 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-04-27T05:22:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.206+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.206+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.206+0000 7f2253f23880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.206+0000 7f2253f23880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55da1cea8600 con 0x55da1cbd1400 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.206+0000 7f2253f23880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55da1cc10280 con 0x55da1cbd1000 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.207+0000 7f2250068640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55da1cbd1400 0x55da1ccd8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:48290/0 (socket says 172.21.15.138:48290) 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.207+0000 7f2250869640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.207+0000 7f2250068640 1 -- 172.21.15.138:0/1194846883 learned_addr learned my addr 172.21.15.138:0/1194846883 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.207+0000 7f2250869640 1 -- 172.21.15.138:0/1194846883 >> v1:172.21.15.114:6789/0 conn(0x55da1cbd1400 legacy=0x55da1ccd8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.207+0000 7f2250869640 1 -- 172.21.15.138:0/1194846883 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cbd5380 con 0x55da1cbd1000 2024-04-27T05:22:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2250869640 1 --2- 172.21.15.138:0/1194846883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55da1ccdcd50 tx=0x55da1cce2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90249684639baf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f224f867640 1 -- 172.21.15.138:0/1194846883 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe9ce0 con 0x55da1cbd1000 2024-04-27T05:22:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f224f867640 1 -- 172.21.15.138:0/1194846883 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1cea9b00 con 0x55da1cbd1000 2024-04-27T05:22:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f224f867640 1 -- 172.21.15.138:0/1194846883 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe9e40 con 0x55da1cbd1000 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 -- 172.21.15.138:0/1194846883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 msgr2=0x55da1ceb2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 --2- 172.21.15.138:0/1194846883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55da1ccdcd50 tx=0x55da1cce2de0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 -- 172.21.15.138:0/1194846883 shutdown_connections 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 --2- 172.21.15.138:0/1194846883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 -- 172.21.15.138:0/1194846883 >> 172.21.15.138:0/1194846883 conn(0x55da1cbd0c00 msgr2=0x55da1ccd8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 -- 172.21.15.138:0/1194846883 shutdown_connections 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.209+0000 7f2253f23880 1 -- 172.21.15.138:0/1194846883 wait complete. 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2253f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2253f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da1cc10780 con 0x55da1cbd1000 2024-04-27T05:22:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2250869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2250869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52360/0 (socket says 172.21.15.138:52360) 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2250869640 1 -- 172.21.15.138:0/3728311232 learned_addr learned my addr 172.21.15.138:0/3728311232 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.210+0000 7f2250869640 1 -- 172.21.15.138:0/3728311232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cbd4ea0 con 0x55da1cbd1000 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2250869640 1 --2- 172.21.15.138:0/3728311232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55da1d9d5560 tx=0x55da1cce2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b852341f5884b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f224f066640 1 -- 172.21.15.138:0/3728311232 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe94a0 con 0x55da1cbd1000 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f224f066640 1 -- 172.21.15.138:0/3728311232 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1cea9e00 con 0x55da1cbd1000 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f224f066640 1 -- 172.21.15.138:0/3728311232 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe9ce0 con 0x55da1cbd1000 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 -- 172.21.15.138:0/3728311232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 msgr2=0x55da1ceb2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 --2- 172.21.15.138:0/3728311232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55da1d9d5560 tx=0x55da1cce2cc0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 -- 172.21.15.138:0/3728311232 shutdown_connections 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 --2- 172.21.15.138:0/3728311232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 -- 172.21.15.138:0/3728311232 >> 172.21.15.138:0/3728311232 conn(0x55da1cbd0c00 msgr2=0x55da1ccd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 -- 172.21.15.138:0/3728311232 shutdown_connections 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 -- 172.21.15.138:0/3728311232 wait complete. 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.211+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2253f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2253f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da1cc108c0 con 0x55da1cbd1000 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2250869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2250869640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52372/0 (socket says 172.21.15.138:52372) 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2250869640 1 -- 172.21.15.138:0/3503528123 learned_addr learned my addr 172.21.15.138:0/3503528123 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2250869640 1 -- 172.21.15.138:0/3503528123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cbd56c0 con 0x55da1cbd1000 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2250869640 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55da1da8cde0 tx=0x55da1cce2c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f224d863640 1 -- 172.21.15.138:0/3503528123 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe9e40 con 0x55da1cbd1000 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55da1cbd51e0 con 0x55da1cbd1000 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f224d863640 1 -- 172.21.15.138:0/3503528123 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1d9b0000 con 0x55da1cbd1000 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.212+0000 7f224d863640 1 -- 172.21.15.138:0/3503528123 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe94a0 con 0x55da1cbd1000 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da1cc42ea0 con 0x55da1cbd1000 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f2253f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f2253f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da1cbf4280 con 0x55da1d9c8c00 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f224d863640 1 -- 172.21.15.138:0/3503528123 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55da1cbfab00 con 0x55da1cbd1000 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f225106a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f225106a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52384/0 (socket says 172.21.15.138:52384) 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f225106a640 1 -- 172.21.15.138:0/234268674 learned_addr learned my addr 172.21.15.138:0/234268674 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f224d863640 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1d9c9c00 0x55da1ceb3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.213+0000 7f224d863640 1 -- 172.21.15.138:0/3503528123 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55da1cbe3a40 con 0x55da1cbd1000 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2250869640 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1d9c9c00 0x55da1ceb3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f225106a640 1 -- 172.21.15.138:0/234268674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cc42d00 con 0x55da1d9c8c00 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f225106a640 1 --2- 172.21.15.138:0/234268674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x55da1dad10e0 tx=0x55da1cce3bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4372ea6f4efefe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2250869640 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1d9c9c00 0x55da1ceb3080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55da1dad02d0 tx=0x55da1cce3d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f224b05e640 1 -- 172.21.15.138:0/234268674 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe9e40 con 0x55da1d9c8c00 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f224b05e640 1 -- 172.21.15.138:0/234268674 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1d9b0600 con 0x55da1d9c8c00 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f224b05e640 1 -- 172.21.15.138:0/234268674 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe94a0 con 0x55da1d9c8c00 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 -- 172.21.15.138:0/234268674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 msgr2=0x55da1ceb2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 --2- 172.21.15.138:0/234268674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x55da1dad10e0 tx=0x55da1cce3bc0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 -- 172.21.15.138:0/234268674 shutdown_connections 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 --2- 172.21.15.138:0/234268674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 -- 172.21.15.138:0/234268674 >> 172.21.15.138:0/234268674 conn(0x55da1cbd1400 msgr2=0x55da1ccdb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 -- 172.21.15.138:0/234268674 shutdown_connections 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.214+0000 7f2253f23880 1 -- 172.21.15.138:0/234268674 wait complete. 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f2253f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f2253f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da1cbf5180 con 0x55da1d9c8c00 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f225106a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f225106a640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52396/0 (socket says 172.21.15.138:52396) 2024-04-27T05:22:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 learned_addr learned my addr 172.21.15.138:0/3406584804 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cc42b60 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.215+0000 7f225106a640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55da1da8ce70 tx=0x55da1cce39e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f224985b640 1 -- 172.21.15.138:0/3406584804 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe9e40 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f224985b640 1 -- 172.21.15.138:0/3406584804 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1d9b0780 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55da1cc429c0 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f224985b640 1 -- 172.21.15.138:0/3406584804 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cbe94a0 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da1cc42820 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f224985b640 1 -- 172.21.15.138:0/3406584804 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55da1cbfb600 con 0x55da1d9c8c00 2024-04-27T05:22:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f224985b640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1cf50000 0x55da1ceb3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.216+0000 7f224985b640 1 -- 172.21.15.138:0/3406584804 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55da1cbe3c00 con 0x55da1d9c8c00 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2250068640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1cf50000 0x55da1ceb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55da1d9c9000 0x55da1ceb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d9c9400 con 0x55da1d9c9000 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f225106a640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55da1d9c9000 0x55da1ceb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2250068640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1cf50000 0x55da1ceb3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55da1db3cf60 tx=0x55da1db32d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f225106a640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55da1d9c9000 0x55da1ceb3b80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 20 realm 2024-04-27T05:22:39.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d9c9800 con 0x55da1d9c9000 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.217+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51000 con 0x55da1d9c9000 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51400 con 0x55da1d9c9000 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55da1cf51800 0x55da1ceb4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51c00 con 0x55da1cf51800 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2250869640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55da1cf51800 0x55da1ceb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2250869640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55da1cf51800 0x55da1ceb4100 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.218+0000 7f2253f23880 20 searching for the correct realm 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51000 con 0x55da1cf51800 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250869640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55da1cf51000 0x55da1ceb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51c00 con 0x55da1cf51000 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250068640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55da1cf51000 0x55da1ceb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250068640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55da1cf51000 0x55da1ceb4c00 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250068640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55da1cf51c00 0x55da1ceb5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.219+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.220+0000 7f225106a640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55da1cf51c00 0x55da1ceb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.220+0000 7f225106a640 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55da1cf51c00 0x55da1ceb5180 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.220+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.220+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.221+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51000 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.222+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51000 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.223+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.224+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51000 2024-04-27T05:22:39.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.225+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.226+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.226+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.226+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.226+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.226+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.226+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51000 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51800 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51c00 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1d9c9000 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf50c00 con 0x55da1cf51000 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.227+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:39.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51400 con 0x55da1d9c9000 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d9c9400 con 0x55da1d9c9000 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d9c9800 con 0x55da1cf51800 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:39.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 zone default found 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 4 Realm: () 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 zonegroup default 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.228+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76000 con 0x55da1cf51c00 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76400 con 0x55da1cf51800 2024-04-27T05:22:39.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76800 con 0x55da1cf51800 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 started sync module instance, tier type = 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.229+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76c00 con 0x55da1cf51000 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.230+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.230+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94395276372224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77000 con 0x55da1cf51000 2024-04-27T05:22:39.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.230+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77400 con 0x55da1d9c9000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.231+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94395276372224] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.232+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.232+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94395276373376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77800 con 0x55da1d9c9000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.232+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77c00 con 0x55da1cf51800 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.233+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94395276373376] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.233+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.233+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94395276374528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76400 con 0x55da1cf51800 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.233+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76800 con 0x55da1cf51000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.234+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94395276374528] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.234+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.234+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94395276375680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77c00 con 0x55da1cf51000 2024-04-27T05:22:39.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.234+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76000 con 0x55da1d9c9000 2024-04-27T05:22:39.269 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.235+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94395276375680] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.236+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.236+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94395276376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d9c9800 con 0x55da1d9c9000 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.236+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d9c9400 con 0x55da1cf51c00 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.237+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94395276376832] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.237+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.237+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94395276377984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf51400 con 0x55da1cf51c00 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.237+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1db2f800 con 0x55da1cf51c00 2024-04-27T05:22:39.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.238+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94395276377984] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.239+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.240+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94395276379136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1db2f400 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.240+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76c00 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94395276379136] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94395276380288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77000 con 0x55da1cf51c00 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=0 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=1 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=2 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=3 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=4 2024-04-27T05:22:39.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=5 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.241+0000 7f2253f23880 20 add_watcher() i=6 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.244+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94395276380288] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.244+0000 7f2253f23880 20 add_watcher() i=7 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.244+0000 7f2253f23880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.244+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf76800 con 0x55da1d9c9000 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.244+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.244+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94395276381440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1cf77400 con 0x55da1d9c9000 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94395276381440] v31'222 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2237837640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2253f23880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2253f23880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:39.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2253f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.245+0000 7f2253f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da1cbf4780 con 0x55da1cf50c00 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2250068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2250068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52410/0 (socket says 172.21.15.138:52410) 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2250068640 1 -- 172.21.15.138:0/1788101426 learned_addr learned my addr 172.21.15.138:0/1788101426 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2250068640 1 -- 172.21.15.138:0/1788101426 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cc42680 con 0x55da1cf50c00 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2250068640 1 --2- 172.21.15.138:0/1788101426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55da1cf30900 tx=0x55da1db32d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9448798df5dedc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2237036640 1 -- 172.21.15.138:0/1788101426 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cf78580 con 0x55da1cf50c00 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2237036640 1 -- 172.21.15.138:0/1788101426 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1d9b0000 con 0x55da1cf50c00 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.246+0000 7f2237036640 1 -- 172.21.15.138:0/1788101426 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cf786e0 con 0x55da1cf50c00 2024-04-27T05:22:39.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- 172.21.15.138:0/1788101426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 msgr2=0x55da1ceb4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 --2- 172.21.15.138:0/1788101426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55da1cf30900 tx=0x55da1db32d20 comp rx=0 tx=0).stop 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- 172.21.15.138:0/1788101426 shutdown_connections 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 --2- 172.21.15.138:0/1788101426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- 172.21.15.138:0/1788101426 >> 172.21.15.138:0/1788101426 conn(0x55da1cf77800 msgr2=0x55da1ccdb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- 172.21.15.138:0/1788101426 shutdown_connections 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- 172.21.15.138:0/1788101426 wait complete. 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 Processor -- start 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- start start 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.247+0000 7f2253f23880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da1cbf4500 con 0x55da1cf50c00 2024-04-27T05:22:39.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2250068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2250068640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52424/0 (socket says 172.21.15.138:52424) 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2250068640 1 -- 172.21.15.138:0/1666551532 learned_addr learned my addr 172.21.15.138:0/1666551532 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2250068640 1 -- 172.21.15.138:0/1666551532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da1cc424e0 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2250068640 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55da1cf31890 tx=0x55da1db331a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2235833640 1 -- 172.21.15.138:0/1666551532 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cf78840 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2235833640 1 -- 172.21.15.138:0/1666551532 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da1d9b0a80 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55da1cbd5380 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2235833640 1 -- 172.21.15.138:0/1666551532 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55da1cf78580 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.248+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da1cbd4ea0 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f2235833640 1 -- 172.21.15.138:0/1666551532 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55da1cbfbb80 con 0x55da1cf50c00 2024-04-27T05:22:39.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f2235833640 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1db2f800 0x55da1ceb5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f2235833640 1 -- 172.21.15.138:0/1666551532 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55da1cbe3dc0 con 0x55da1cf50c00 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f2253f23880 5 note: GC not initialized 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f225106a640 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1db2f800 0x55da1ceb5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f223302e640 20 reqs_thread_entry: start 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.249+0000 7f2253f23880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f225106a640 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1db2f800 0x55da1ceb5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55da1cf31200 tx=0x55da1db33020 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f223102a640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:09 2024 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2230808640 20 reqs_thread_entry: start 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T05:22:39.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04a000 con 0x55da1cf51000 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.250+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04a400 con 0x55da1cf51c00 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.251+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 fud [create,call,writefull 0~181] v31'6 uv5 ondisk = -17 ((17) File exists)) v8 ==== 231+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.252+0000 7f2237837640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.252+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94395276381440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04a800 con 0x55da1d9c9000 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.253+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94395276381440] v31'223 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.253+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94395276372224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04ac00 con 0x55da1cf51000 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.254+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94395276372224] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.254+0000 7f2253f23880 20 remove_watcher() i=0 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.254+0000 7f2253f23880 2 removed watcher, disabling cache 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.254+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94395276373376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04b000 con 0x55da1d9c9000 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.255+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94395276373376] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.255+0000 7f2253f23880 20 remove_watcher() i=1 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.255+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94395276374528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04b400 con 0x55da1cf51800 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.256+0000 7f2250869640 1 -- 172.21.15.138:0/3406584804 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94395276374528] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64fc0 con 0x55da1cf51800 2024-04-27T05:22:39.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.256+0000 7f2253f23880 20 remove_watcher() i=2 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.256+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94395276375680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04b800 con 0x55da1cf51000 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.257+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94395276375680] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf65200 con 0x55da1cf51000 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.257+0000 7f2253f23880 20 remove_watcher() i=3 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.257+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94395276376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1d04bc00 con 0x55da1d9c9000 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.258+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94395276376832] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1d9c9000 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.258+0000 7f2253f23880 20 remove_watcher() i=4 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.258+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94395276377984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1db56000 con 0x55da1cf51c00 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.259+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94395276377984] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.259+0000 7f2253f23880 20 remove_watcher() i=5 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.259+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94395276379136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1db56400 con 0x55da1cf51c00 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.260+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94395276379136] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.260+0000 7f2253f23880 20 remove_watcher() i=6 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.260+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94395276380288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55da1db56800 con 0x55da1cf51c00 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f225106a640 1 -- 172.21.15.138:0/3406584804 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94395276380288] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55da1cf64900 con 0x55da1cf51c00 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 20 remove_watcher() i=7 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55da1cf51c00 msgr2=0x55da1ceb5180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55da1cf51c00 0x55da1ceb5180 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55da1d9c9000 msgr2=0x55da1ceb3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55da1d9c9000 0x55da1ceb3b80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55da1cf51000 msgr2=0x55da1ceb4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55da1cf51000 0x55da1ceb4c00 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55da1cf51800 msgr2=0x55da1ceb4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.261+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55da1cf51800 0x55da1ceb4100 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1cf50000 msgr2=0x55da1ceb3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1cf50000 0x55da1ceb3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55da1db3cf60 tx=0x55da1db32d80 comp rx=0 tx=0).stop 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 msgr2=0x55da1ceb2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/3406584804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1d9c8c00 0x55da1ceb2b00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55da1da8ce70 tx=0x55da1cce39e0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 reap_dead start 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2250068640 1 -- 172.21.15.138:0/3406584804 reap_dead start 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 shutdown_connections 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 >> 172.21.15.138:0/3406584804 conn(0x55da1cbd1400 msgr2=0x55da1ccdb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 shutdown_connections 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3406584804 wait complete. 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1db2f800 msgr2=0x55da1ceb5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1db2f800 0x55da1ceb5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55da1cf31200 tx=0x55da1db33020 comp rx=0 tx=0).stop 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 msgr2=0x55da1ceb4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55da1cf31890 tx=0x55da1db331a0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 shutdown_connections 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1db2f800 0x55da1ceb5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/1666551532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cf50c00 0x55da1ceb4680 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 >> 172.21.15.138:0/1666551532 conn(0x55da1cf77800 msgr2=0x55da1ccdb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 shutdown_connections 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/1666551532 wait complete. 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1d9c9c00 msgr2=0x55da1ceb3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1d9c9c00 0x55da1ceb3080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55da1dad02d0 tx=0x55da1cce3d40 comp rx=0 tx=0).stop 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 msgr2=0x55da1ceb2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.262+0000 7f2253f23880 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55da1da8cde0 tx=0x55da1cce2c60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.263+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 shutdown_connections 2024-04-27T05:22:39.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.263+0000 7f2253f23880 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55da1d9c9c00 0x55da1ceb3080 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.263+0000 7f2253f23880 1 --2- 172.21.15.138:0/3503528123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da1cbd1000 0x55da1ceb2580 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.263+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 >> 172.21.15.138:0/3503528123 conn(0x55da1cbd0c00 msgr2=0x55da1ccd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.263+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 shutdown_connections 2024-04-27T05:22:39.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.263+0000 7f2253f23880 1 -- 172.21.15.138:0/3503528123 wait complete. 2024-04-27T05:22:39.287 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:22:39.287 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-27T05:22:39.292 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-27T05:22:39.292 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-04-27T05:22:39.292 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-27T05:22:39.292 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-27T05:22:39.293 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-04-27T05:22:39.336 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:39.336 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:39.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573b8bb880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573b8bb880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b0b6ade600 con 0x55b0b6807400 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573b8bb880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b0b6846280 con 0x55b0b6807000 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573a060640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573985f640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55b0b6807400 0x55b0b690e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:48306/0 (socket says 172.21.15.138:48306) 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573a060640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52430/0 (socket says 172.21.15.138:52430) 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.335+0000 7f573a060640 1 -- 172.21.15.138:0/1243154796 learned_addr learned my addr 172.21.15.138:0/1243154796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.336+0000 7f573a060640 1 -- 172.21.15.138:0/1243154796 >> v1:172.21.15.114:6789/0 conn(0x55b0b6807400 legacy=0x55b0b690e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.336+0000 7f573a060640 1 -- 172.21.15.138:0/1243154796 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b680b380 con 0x55b0b6807000 2024-04-27T05:22:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.337+0000 7f573a060640 1 --2- 172.21.15.138:0/1243154796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55b0b6912d50 tx=0x55b0b6918de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bf1e6c2023a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.337+0000 7f573905e640 1 -- 172.21.15.138:0/1243154796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681fce0 con 0x55b0b6807000 2024-04-27T05:22:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.337+0000 7f573905e640 1 -- 172.21.15.138:0/1243154796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b6adfb00 con 0x55b0b6807000 2024-04-27T05:22:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.337+0000 7f573905e640 1 -- 172.21.15.138:0/1243154796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681fe40 con 0x55b0b6807000 2024-04-27T05:22:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 -- 172.21.15.138:0/1243154796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 msgr2=0x55b0b6ae8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 --2- 172.21.15.138:0/1243154796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55b0b6912d50 tx=0x55b0b6918de0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 -- 172.21.15.138:0/1243154796 shutdown_connections 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 --2- 172.21.15.138:0/1243154796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 -- 172.21.15.138:0/1243154796 >> 172.21.15.138:0/1243154796 conn(0x55b0b6806c00 msgr2=0x55b0b690e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 -- 172.21.15.138:0/1243154796 shutdown_connections 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.338+0000 7f573b8bb880 1 -- 172.21.15.138:0/1243154796 wait complete. 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573b8bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573b8bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b6846780 con 0x55b0b6807000 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573a060640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573a060640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52442/0 (socket says 172.21.15.138:52442) 2024-04-27T05:22:39.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573a060640 1 -- 172.21.15.138:0/283218621 learned_addr learned my addr 172.21.15.138:0/283218621 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573a060640 1 -- 172.21.15.138:0/283218621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b680aea0 con 0x55b0b6807000 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573a060640 1 --2- 172.21.15.138:0/283218621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55b0b760b560 tx=0x55b0b6918cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e21658f0ca56c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573885d640 1 -- 172.21.15.138:0/283218621 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681f4a0 con 0x55b0b6807000 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573885d640 1 -- 172.21.15.138:0/283218621 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b6adfe00 con 0x55b0b6807000 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.339+0000 7f573885d640 1 -- 172.21.15.138:0/283218621 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681fce0 con 0x55b0b6807000 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- 172.21.15.138:0/283218621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 msgr2=0x55b0b6ae8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 --2- 172.21.15.138:0/283218621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x55b0b760b560 tx=0x55b0b6918cc0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- 172.21.15.138:0/283218621 shutdown_connections 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 --2- 172.21.15.138:0/283218621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- 172.21.15.138:0/283218621 >> 172.21.15.138:0/283218621 conn(0x55b0b6806c00 msgr2=0x55b0b690f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- 172.21.15.138:0/283218621 shutdown_connections 2024-04-27T05:22:39.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- 172.21.15.138:0/283218621 wait complete. 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573b8bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b68468c0 con 0x55b0b6807000 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.340+0000 7f573a060640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573a060640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52444/0 (socket says 172.21.15.138:52444) 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573a060640 1 -- 172.21.15.138:0/1228124652 learned_addr learned my addr 172.21.15.138:0/1228124652 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573a060640 1 -- 172.21.15.138:0/1228124652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b680b6c0 con 0x55b0b6807000 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573a060640 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x55b0b76c32c0 tx=0x55b0b6918c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573705a640 1 -- 172.21.15.138:0/1228124652 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681fe40 con 0x55b0b6807000 2024-04-27T05:22:39.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573705a640 1 -- 172.21.15.138:0/1228124652 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b75e6000 con 0x55b0b6807000 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0b680b1e0 con 0x55b0b6807000 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573705a640 1 -- 172.21.15.138:0/1228124652 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681f4a0 con 0x55b0b6807000 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.341+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0b6878ea0 con 0x55b0b6807000 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573705a640 1 -- 172.21.15.138:0/1228124652 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b0b6830b00 con 0x55b0b6807000 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573b8bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573b8bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b682a280 con 0x55b0b75fec00 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573705a640 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b75ffc00 0x55b0b6ae9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573705a640 1 -- 172.21.15.138:0/1228124652 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55b0b6819a40 con 0x55b0b6807000 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a861640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a861640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52460/0 (socket says 172.21.15.138:52460) 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a861640 1 -- 172.21.15.138:0/3943834675 learned_addr learned my addr 172.21.15.138:0/3943834675 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a060640 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b75ffc00 0x55b0b6ae9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a861640 1 -- 172.21.15.138:0/3943834675 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b6878d00 con 0x55b0b75fec00 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a060640 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b75ffc00 0x55b0b6ae9080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55b0b7708180 tx=0x55b0b6919c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.342+0000 7f573a861640 1 --2- 172.21.15.138:0/3943834675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55b0b76c3e60 tx=0x55b0b6919f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bffff0a9784c5be server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f5734855640 1 -- 172.21.15.138:0/3943834675 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681fe40 con 0x55b0b75fec00 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f5734855640 1 -- 172.21.15.138:0/3943834675 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b75e6600 con 0x55b0b75fec00 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f5734855640 1 -- 172.21.15.138:0/3943834675 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681f4a0 con 0x55b0b75fec00 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- 172.21.15.138:0/3943834675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 msgr2=0x55b0b6ae8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 --2- 172.21.15.138:0/3943834675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55b0b76c3e60 tx=0x55b0b6919f80 comp rx=0 tx=0).stop 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- 172.21.15.138:0/3943834675 shutdown_connections 2024-04-27T05:22:39.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 --2- 172.21.15.138:0/3943834675 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- 172.21.15.138:0/3943834675 >> 172.21.15.138:0/3943834675 conn(0x55b0b6807400 msgr2=0x55b0b6911000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- 172.21.15.138:0/3943834675 shutdown_connections 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- 172.21.15.138:0/3943834675 wait complete. 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.343+0000 7f573b8bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b682b180 con 0x55b0b75fec00 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573a861640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573a861640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52476/0 (socket says 172.21.15.138:52476) 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 learned_addr learned my addr 172.21.15.138:0/2571899439 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b6878b60 con 0x55b0b75fec00 2024-04-27T05:22:39.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573a861640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55b0b76c2b40 tx=0x55b0b6919b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f5733052640 1 -- 172.21.15.138:0/2571899439 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681fe40 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f5733052640 1 -- 172.21.15.138:0/2571899439 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b75e6780 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0b68789c0 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f5733052640 1 -- 172.21.15.138:0/2571899439 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b681f4a0 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.344+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0b6878820 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f5733052640 1 -- 172.21.15.138:0/2571899439 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b0b6831600 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f5733052640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b6b80000 0x55b0b6ae9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f5733052640 1 -- 172.21.15.138:0/2571899439 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55b0b6819c00 con 0x55b0b75fec00 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f573985f640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b6b80000 0x55b0b6ae9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b75ff000 0x55b0b6ae9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b75ff000 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f573a861640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b75ff000 0x55b0b6ae9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.345+0000 7f573985f640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b6b80000 0x55b0b6ae9600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55b0b6913380 tx=0x55b0b776ef60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573a861640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b75ff000 0x55b0b6ae9b80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 realm 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff800 con 0x55b0b75ff000 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:39.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81000 con 0x55b0b75ff000 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81400 con 0x55b0b75ff000 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.346+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6b81800 0x55b0b6aea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81c00 con 0x55b0b6b81800 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573a060640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6b81800 0x55b0b6aea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573a060640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6b81800 0x55b0b6aea100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573b8bb880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573b8bb880 20 searching for the correct realm 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81000 con 0x55b0b6b81800 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573a060640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6b81000 0x55b0b6aeac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.347+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81c00 con 0x55b0b6b81000 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.348+0000 7f573985f640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6b81000 0x55b0b6aeac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.348+0000 7f573985f640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6b81000 0x55b0b6aeac00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.348+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.348+0000 7f573985f640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6b81c00 0x55b0b6aeb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.348+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.348+0000 7f573a861640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6b81c00 0x55b0b6aeb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6b81c00 0x55b0b6aeb180 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.349+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.350+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81000 2024-04-27T05:22:39.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.351+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.352+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.353+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.354+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81000 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81800 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b75ff000 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.355+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b80c00 con 0x55b0b6b81000 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81400 con 0x55b0b75ff000 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b75ff000 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff800 con 0x55b0b6b81800 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 zone default found 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 4 Realm: () 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:39.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 zonegroup default 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.356+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa000 con 0x55b0b6b81c00 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa400 con 0x55b0b6b81800 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa800 con 0x55b0b6b81800 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 started sync module instance, tier type = 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.357+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baac00 con 0x55b0b6b81000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.358+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.358+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94217467463936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab000 con 0x55b0b6b81000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.358+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab400 con 0x55b0b75ff000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.359+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94217467463936] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.359+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.359+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94217467465088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab800 con 0x55b0b75ff000 2024-04-27T05:22:39.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.359+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6babc00 con 0x55b0b6b81800 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.360+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94217467465088] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.361+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.361+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94217467466240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa400 con 0x55b0b6b81800 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.361+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa800 con 0x55b0b6b81000 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.361+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.361+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94217467466240] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.362+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94217467467392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6babc00 con 0x55b0b6b81000 2024-04-27T05:22:39.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.362+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa000 con 0x55b0b75ff000 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.363+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.363+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94217467468544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff800 con 0x55b0b75ff000 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.363+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b6b81c00 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.363+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94217467467392] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.364+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.364+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94217467469696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6b81400 con 0x55b0b6b81c00 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.364+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b7764c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.364+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94217467468544] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.364+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94217467469696] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.365+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.365+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94217467470848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b7764800 con 0x55b0b6b81c00 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.365+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baac00 con 0x55b0b6b81c00 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94217467470848] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94217467472000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab000 con 0x55b0b6b81c00 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=0 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=1 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=2 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=3 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=4 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=5 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.366+0000 7f573b8bb880 20 add_watcher() i=6 2024-04-27T05:22:39.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.367+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94217467472000] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.367+0000 7f573b8bb880 20 add_watcher() i=7 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.367+0000 7f573b8bb880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.367+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa800 con 0x55b0b75ff000 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.367+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.367+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94217467473152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab400 con 0x55b0b75ff000 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94217467473152] v31'224 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f572102e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573b8bb880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573b8bb880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573b8bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.368+0000 7f573b8bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b682a780 con 0x55b0b6b80c00 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f573985f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f573985f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52482/0 (socket says 172.21.15.138:52482) 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f573985f640 1 -- 172.21.15.138:0/4181497841 learned_addr learned my addr 172.21.15.138:0/4181497841 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f573985f640 1 -- 172.21.15.138:0/4181497841 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b6878680 con 0x55b0b6b80c00 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f573985f640 1 --2- 172.21.15.138:0/4181497841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55b0b6b62ae0 tx=0x55b0b776ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbb077082e12454 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f572082d640 1 -- 172.21.15.138:0/4181497841 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b6bac580 con 0x55b0b6b80c00 2024-04-27T05:22:39.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f572082d640 1 -- 172.21.15.138:0/4181497841 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b75e6000 con 0x55b0b6b80c00 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.369+0000 7f572082d640 1 -- 172.21.15.138:0/4181497841 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b6bac6e0 con 0x55b0b6b80c00 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- 172.21.15.138:0/4181497841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 msgr2=0x55b0b6aea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 --2- 172.21.15.138:0/4181497841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55b0b6b62ae0 tx=0x55b0b776ef00 comp rx=0 tx=0).stop 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- 172.21.15.138:0/4181497841 shutdown_connections 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 --2- 172.21.15.138:0/4181497841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- 172.21.15.138:0/4181497841 >> 172.21.15.138:0/4181497841 conn(0x55b0b6bab800 msgr2=0x55b0b6911800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- 172.21.15.138:0/4181497841 shutdown_connections 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- 172.21.15.138:0/4181497841 wait complete. 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 Processor -- start 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- start start 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.370+0000 7f573b8bb880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0b682a500 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573985f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573985f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52492/0 (socket says 172.21.15.138:52492) 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 learned_addr learned my addr 172.21.15.138:0/155823548 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0b68784e0 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573985f640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55b0b6b63800 tx=0x55b0b776f080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f571f02a640 1 -- 172.21.15.138:0/155823548 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b6bac840 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f571f02a640 1 -- 172.21.15.138:0/155823548 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0b75e6a80 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f571f02a640 1 -- 172.21.15.138:0/155823548 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55b0b6bac580 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0b680b380 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.371+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0b680aea0 con 0x55b0b6b80c00 2024-04-27T05:22:39.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f571f02a640 1 -- 172.21.15.138:0/155823548 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55b0b6831b80 con 0x55b0b6b80c00 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f571f02a640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b7764c00 0x55b0b6aeb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f571f02a640 1 -- 172.21.15.138:0/155823548 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55b0b6819dc0 con 0x55b0b6b80c00 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f573a861640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b7764c00 0x55b0b6aeb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b7764800 0x55b0b6c48000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c000 con 0x55b0b7764800 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.372+0000 7f573a060640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b7764800 0x55b0b6c48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.373+0000 7f573a861640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b7764c00 0x55b0b6aeb700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55b0b77701b0 tx=0x55b0b776fe60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.373+0000 7f573a060640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b7764800 0x55b0b6c48000 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.374+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'60 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b7764800 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.374+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6c4c400 0x55b0b6c48580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.374+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c800 con 0x55b0b6c4c400 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.374+0000 7f573985f640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6c4c400 0x55b0b6c48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.374+0000 7f573985f640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6c4c400 0x55b0b6c48580 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.375+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.375+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6c4cc00 0x55b0b6c48b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.375+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d000 con 0x55b0b6c4cc00 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.375+0000 7f573a861640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6c4cc00 0x55b0b6c48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.375+0000 7f573a861640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6c4cc00 0x55b0b6c48b00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.376+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.376+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b7764800 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.377+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'225 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b7764800 2024-04-27T05:22:39.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.377+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6c4d800 0x55b0b6c49080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.377+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4dc00 con 0x55b0b6c4d800 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.378+0000 7f573a060640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6c4d800 0x55b0b6c49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.378+0000 7f573a060640 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6c4d800 0x55b0b6c49080 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.379+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'74 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.379+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64000 con 0x55b0b6c4cc00 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.380+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'60 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.380+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b7764800 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.381+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'35 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b7764800 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.381+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6c4c400 2024-04-27T05:22:39.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.382+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'51 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.382+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64c00 con 0x55b0b6c4d800 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.382+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.382+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65000 con 0x55b0b6c4cc00 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.383+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.383+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b6c4d800 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.384+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'75 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.384+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65800 con 0x55b0b6c4c400 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.385+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'33 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.385+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65c00 con 0x55b0b6c4d800 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.386+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.386+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b6c4c400 2024-04-27T05:22:39.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.387+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'68 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.387+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64c00 con 0x55b0b7764800 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.388+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'226 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b7764800 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.388+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b6c4cc00 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.389+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'74 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.389+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c000 con 0x55b0b6c4d800 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.390+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'76 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.390+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b6c4cc00 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.391+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'38 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.391+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4dc00 con 0x55b0b7764800 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.392+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b7764800 2024-04-27T05:22:39.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.392+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa400 con 0x55b0b7764800 2024-04-27T05:22:39.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.393+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'227 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b7764800 2024-04-27T05:22:39.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.393+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64000 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.394+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'75 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.394+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65000 con 0x55b0b6c4d800 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.395+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'42 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.395+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.396+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.396+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.397+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'44 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.397+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65800 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.397+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.398+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b6c4cc00 2024-04-27T05:22:39.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.398+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'61 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.398+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d000 con 0x55b0b6c4c400 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.399+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'35 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.399+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b6c4cc00 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.400+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'38 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.400+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c800 con 0x55b0b6c4c400 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.401+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'52 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.401+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b6c4cc00 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.402+0000 7f573a861640 1 -- 172.21.15.138:0/155823548 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'76 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6c4cc00 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.402+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baac00 con 0x55b0b6c4c400 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.403+0000 7f573985f640 1 -- 172.21.15.138:0/155823548 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'69 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6c4c400 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.403+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab000 con 0x55b0b6c4d800 2024-04-27T05:22:39.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.404+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.404+0000 7f571c825640 20 reqs_thread_entry: start 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.404+0000 7f573b8bb880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f571a821640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:09 2024 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f5719fff640 20 reqs_thread_entry: start 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca000 con 0x55b0b6b81000 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:22:39.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca400 con 0x55b0b75ff000 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 bud.buckets [call out=184b] v0'0 uv6 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 : miss 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.405+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 8.5 8:a5333f3b:root::.bucket.meta.myfoo6%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca800 con 0x55b0b75ff000 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 .bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=212b] v0'0 uv5 ondisk = 0) v8 ==== 333+0+629 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 20 rados_obj.operate() r=0 bl.length=353 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x17 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:39.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 4.1a 4:5aa84941:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccac00 con 0x55b0b6b81000 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(65 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.406+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:66 4.23 4:c42703a1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb000 con 0x55b0b6b81c00 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.407+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.407+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 4.38 4:1d132ca1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb400 con 0x55b0b75ff000 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.407+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.407+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 4.1e 4:78bf5de1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb800 con 0x55b0b6b81c00 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.408+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.408+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 4.1c 4:395ac92c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccbc00 con 0x55b0b6b81c00 2024-04-27T05:22:39.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.408+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.408+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 4.1c 4:3a8f09fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca400 con 0x55b0b6b81c00 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.408+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.408+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:71 4.19 4:9aea267d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b6b81800 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.2b 4:d4d5e5db:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b75ff000 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 4.6 4:630ee69a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64000 con 0x55b0b6b81800 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b75ff000 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.409+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [call out=289b] v0'0 uv13 ondisk = 0) v8 ==== 194+0+289 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 10 cls_bucket_list_unordered: got thirteen 2024-04-27T05:22:39.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:75 4.11 4:89256115:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6b81c00 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 20 get_obj_state: rctx=0x55b0b773b340 obj=myfoo6:thirteen state=0x55b0b6cd01e8 s->prefetch_data=0 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:76 3.9 3:90a65acf:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65800 con 0x55b0b6b81800 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(76 2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen [getxattrs out=820b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 238+0+836 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 10 manifest: total_size = 8 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 20 get_obj_state: setting s->obj_tag to _oLdIDBlRFwcehRVAOc-MCn1mR-r1QBV 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 20 get_obj_state: rctx=0x55b0b773b340 obj=myfoo6:thirteen state=0x55b0b6cd01e8 s->prefetch_data=0 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 20 prepare_atomic_modification: state is not atomic. state=0x55b0b6cd01e8 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.410+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b75ff000 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.411+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(77 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [stat,call,call] v31'17 uv17 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.411+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:78 3.9 3:90a65acf:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b6b81800 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(78 2f19b8ff-2608-4d86-8f1c-430623205476.4231.7_thirteen [call] v31'3 uv3 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab000 con 0x55b0b75ff000 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 20 list_objects_ordered: starting attempt 1 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:80 4.1a 4:5aa84941:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b6b81000 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:81 4.23 4:c42703a1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6b81c00 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:82 4.38 4:1d132ca1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65c00 con 0x55b0b75ff000 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 4.1e 4:78bf5de1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 4.1c 4:395ac92c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65000 con 0x55b0b6b81c00 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:85 4.1c 4:3a8f09fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b6b81c00 2024-04-27T05:22:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.412+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:86 4.19 4:9aea267d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c000 con 0x55b0b6b81800 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:87 4.2b 4:d4d5e5db:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4dc00 con 0x55b0b75ff000 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:88 4.6 4:630ee69a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca800 con 0x55b0b6b81800 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb000 con 0x55b0b75ff000 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:90 4.11 4:89256115:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb400 con 0x55b0b6b81c00 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(80 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(81 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(83 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(84 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(90 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(85 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(88 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(82 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(87 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.413+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(86 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(79 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [stat,call,call] v31'18 uv18 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f5734054640 20 handle_completion(): completion ok for obj=thirteen 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(89 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [call out=124b] v0'0 uv18 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7])"; uploads.size()=0; is_truncated=0 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]) 2024-04-27T05:22:39.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:91 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb800 con 0x55b0b75ff000 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(91 myfoo6 [call out=48b,stat out=16b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 276+0+305 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 lifecycle: adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.414+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:92 8.5 8:a5333f3b:root::.bucket.meta.myfoo6%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca400 con 0x55b0b75ff000 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.415+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(92 .bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v31'7 uv7 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.415+0000 7f573b8bb880 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x17 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.415+0000 7f573b8bb880 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:39.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.415+0000 7f573b8bb880 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.415+0000 7f573b8bb880 10 lifecycle: distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7, ofs0, ns] 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.415+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:93 7.3 7:c609908c:::notify.5:head [notify cookie 94217467476608 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb400 con 0x55b0b6b81c00 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== watch-notify(notify (1) cookie 94217467469696 notify 133143986209 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x55b0b6c03a00 con 0x55b0b6b81c00 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f5734054640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986209 cookie 94217467469696 notifier 4829 bl.length()=721 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f5734054640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x17 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(93 notify.5 [notify cookie 94217467476608 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f5734054640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f5734054640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f5734054640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa400 con 0x55b0b6b81c00 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(94 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== watch-notify(notify_complete (2) cookie 94217467476608 notify 133143986209 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b0b6c03ba0 con 0x55b0b6b81c00 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573b8bb880 20 lifecycle: handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): orig_sources= new_sources= 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573b8bb880 20 lifecycle: handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): potential sources added=[] removed=[] 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573b8bb880 20 lifecycle: handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): orig_dests= new_dests= 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573b8bb880 20 lifecycle: handle_bi_update(): bucket=:myfoo6[2f19b8ff-2608-4d86-8f1c-430623205476.4231.7]): potential dests added=[] removed=[] 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.416+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 6.14 6:2e7dc4d8:lc::lc.22:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b6c4d800 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.417+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(33 lc.22 [call] v31'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.417+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:34 6.14 6:2e7dc4d8:lc::lc.22:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4dc00 con 0x55b0b6c4d800 2024-04-27T05:22:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.418+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(34 lc.22 [call] v31'25 uv25 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.418+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:35 6.14 6:2e7dc4d8:lc::lc.22:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b6c4d800 2024-04-27T05:22:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(35 lc.22 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6c4d800 2024-04-27T05:22:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:95 4.1a 4:5aa84941:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6b81000 2024-04-27T05:22:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:96 4.23 4:c42703a1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65c00 con 0x55b0b6b81c00 2024-04-27T05:22:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:97 4.38 4:1d132ca1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64c00 con 0x55b0b75ff000 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:98 4.1e 4:78bf5de1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65000 con 0x55b0b6b81c00 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:99 4.1c 4:395ac92c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab000 con 0x55b0b6b81c00 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:100 4.1c 4:3a8f09fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b6b81c00 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:101 4.19 4:9aea267d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6b81800 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:102 4.2b 4:d4d5e5db:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca000 con 0x55b0b75ff000 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:103 4.6 4:630ee69a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccac00 con 0x55b0b6b81800 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:104 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baac00 con 0x55b0b75ff000 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:105 4.11 4:89256115:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d000 con 0x55b0b6b81c00 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(95 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== osd_op_reply(96 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== osd_op_reply(98 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 33 ==== osd_op_reply(99 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(101 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(97 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 34 ==== osd_op_reply(105 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 35 ==== osd_op_reply(100 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(104 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [call out=124b] v0'0 uv18 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(102 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(103 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.419+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:106 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c800 con 0x55b0b75ff000 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.420+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 36 ==== osd_op_reply(106 bud.buckets [call] v31'7 uv7 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.420+0000 7f573b8bb880 10 cache get: name=default.rgw.meta+root+myfoo6 : hit (requested=0x11, cached=0x17) 2024-04-27T05:22:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.420+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:107 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa400 con 0x55b0b75ff000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.421+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 37 ==== osd_op_reply(107 myfoo6 [call,call,delete] v31'19 uv19 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.421+0000 7f573b8bb880 10 removing default.rgw.meta+root+myfoo6 from cache 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.421+0000 7f573b8bb880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.421+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 94217467476608 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6bab000 con 0x55b0b6b81000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.421+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== watch-notify(notify (1) cookie 94217467467392 notify 133143986194 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x55b0b6c03d40 con 0x55b0b6b81000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.422+0000 7f5734054640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986194 cookie 94217467467392 notifier 4829 bl.length()=150 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.422+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(108 notify.3 [notify cookie 94217467476608 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.422+0000 7f5734054640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64000 con 0x55b0b6b81000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.422+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.422+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== watch-notify(notify_complete (2) cookie 94217467476608 notify 133143986194 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b0b6bff860 con 0x55b0b6b81000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.422+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:110 8.5 8:a5333f3b:root::.bucket.meta.myfoo6%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca800 con 0x55b0b75ff000 2024-04-27T05:22:39.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 38 ==== osd_op_reply(110 .bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [call,call,delete] v31'8 uv8 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573b8bb880 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 from cache 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573b8bb880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:2f19b8ff-2608-4d86-8f1c-430623205476.4231.7, ofs0, ns] 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [notify cookie 94217467476608 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb800 con 0x55b0b6b81c00 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 36 ==== watch-notify(notify (1) cookie 94217467469696 notify 133143986210 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x55b0b6c03ba0 con 0x55b0b6b81c00 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f5734054640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986210 cookie 94217467469696 notifier 4829 bl.length()=207 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f5734054640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b6b81c00 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 37 ==== osd_op_reply(111 notify.5 [notify cookie 94217467476608 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.423+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 38 ==== osd_op_reply(112 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 39 ==== watch-notify(notify_complete (2) cookie 94217467476608 notify 133143986210 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b0b6c021a0 con 0x55b0b6b81c00 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:113 4.1a 4:5aa84941:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb000 con 0x55b0b6b81000 2024-04-27T05:22:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:114 4.23 4:c42703a1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccbc00 con 0x55b0b6b81c00 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:115 4.38 4:1d132ca1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca400 con 0x55b0b75ff000 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:116 4.1e 4:78bf5de1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb400 con 0x55b0b6b81c00 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:117 4.1c 4:395ac92c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c000 con 0x55b0b6b81c00 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:118 4.1c 4:3a8f09fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d400 con 0x55b0b6b81c00 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:119 4.19 4:9aea267d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4dc00 con 0x55b0b6b81800 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:120 4.2b 4:d4d5e5db:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b75ff400 con 0x55b0b75ff000 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:121 4.6 4:630ee69a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64400 con 0x55b0b6b81800 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:122 4.9 4:92ecab8f:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64c00 con 0x55b0b75ff000 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.424+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:123 4.11 4:89256115:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65000 con 0x55b0b6b81c00 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(113 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.0 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 40 ==== osd_op_reply(117 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.4 [delete] v31'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 39 ==== osd_op_reply(115 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.2 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 41 ==== osd_op_reply(114 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.1 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 42 ==== osd_op_reply(118 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.5 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 40 ==== osd_op_reply(120 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.7 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 43 ==== osd_op_reply(116 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.3 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.425+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 44 ==== osd_op_reply(123 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.10 [delete] v31'4 uv4 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 41 ==== osd_op_reply(122 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.9 [delete] v31'19 uv19 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(119 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.6 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(121 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.7.8 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 : miss 2024-04-27T05:22:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:124 6.c 6:303fa32c:::pubsub..bucket.myfoo6%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65400 con 0x55b0b6b81800 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== osd_op_reply(124 pubsub..bucket.myfoo6/2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 info.flags=0x0 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 10 adding default.rgw.log++pubsub..bucket.myfoo6/2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 to cache LRU end 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.426+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:125 6.c 6:303fa32c:::pubsub..bucket.myfoo6%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65800 con 0x55b0b6b81800 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.427+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(125 pubsub..bucket.myfoo6/2f19b8ff-2608-4d86-8f1c-430623205476.4231.7 [delete] v31'11 uv10 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.427+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:126 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c65c00 con 0x55b0b75ff000 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.428+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 42 ==== osd_op_reply(126 bud.buckets [call] v31'8 uv8 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.428+0000 7f573b8bb880 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.428+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:127 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4d000 con 0x55b0b75ff000 2024-04-27T05:22:39.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.428+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 43 ==== osd_op_reply(127 NX5QOQKC6BH2IDN8HC7A [delete] v31'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.428+0000 7f573b8bb880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.428+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:128 7.3 7:c609908c:::notify.5:head [notify cookie 94217467476608 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c800 con 0x55b0b6b81c00 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 45 ==== watch-notify(notify (1) cookie 94217467469696 notify 133143986211 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55b0b6c021a0 con 0x55b0b6b81c00 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f5734054640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986211 cookie 94217467469696 notifier 4829 bl.length()=170 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f5734054640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:129 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c64800 con 0x55b0b6b81c00 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 46 ==== osd_op_reply(128 notify.5 [notify cookie 94217467476608 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 47 ==== osd_op_reply(129 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 48 ==== watch-notify(notify_complete (2) cookie 94217467476608 notify 133143986211 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b0b6c02340 con 0x55b0b6b81c00 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573b8bb880 10 removing email index: 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573b8bb880 10 removing user buckets index 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.429+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:130 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6c4c000 con 0x55b0b75ff000 2024-04-27T05:22:39.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 44 ==== osd_op_reply(130 bud.buckets [delete] v31'9 uv9 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f573b8bb880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:131 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94217467476608 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca000 con 0x55b0b6b81c00 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 49 ==== watch-notify(notify (1) cookie 94217467470848 notify 133143986212 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55b0b6c02340 con 0x55b0b6b81c00 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f5734855640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986212 cookie 94217467470848 notifier 4829 bl.length()=160 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f5734855640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:132 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cea000 con 0x55b0b6b81c00 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.430+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 50 ==== osd_op_reply(131 notify.6 [notify cookie 94217467476608 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.431+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 51 ==== osd_op_reply(132 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.431+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 52 ==== watch-notify(notify_complete (2) cookie 94217467476608 notify 133143986212 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b0b6c024e0 con 0x55b0b6b81c00 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.431+0000 7f573b8bb880 10 removing user index: bud 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.431+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:133 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca800 con 0x55b0b6b81000 2024-04-27T05:22:39.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(133 bud [call,call,delete] v31'6 uv6 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573b8bb880 10 removing default.rgw.meta+users.uid+bud from cache 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573b8bb880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94217467476608 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb800 con 0x55b0b6b81c00 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 53 ==== watch-notify(notify (1) cookie 94217467469696 notify 133143986213 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55b0b6c024e0 con 0x55b0b6b81c00 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f5734054640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986213 cookie 94217467469696 notifier 4829 bl.length()=152 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 54 ==== osd_op_reply(134 notify.5 [notify cookie 94217467476608 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f5734054640 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cea800 con 0x55b0b6b81c00 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 55 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.432+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 56 ==== watch-notify(notify_complete (2) cookie 94217467476608 notify 133143986213 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55b0b6c02680 con 0x55b0b6b81c00 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.433+0000 7f572102e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.433+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:136 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94217467473152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccbc00 con 0x55b0b75ff000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.434+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 45 ==== osd_op_reply(136 data_loggenerations_metadata [watch unwatch cookie 94217467473152] v31'228 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.434+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:137 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94217467463936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cca400 con 0x55b0b6b81000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.435+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(137 notify.0 [watch unwatch cookie 94217467463936] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.435+0000 7f573b8bb880 20 remove_watcher() i=0 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.435+0000 7f573b8bb880 2 removed watcher, disabling cache 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.435+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:138 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94217467465088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ccb400 con 0x55b0b75ff000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.436+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 46 ==== osd_op_reply(138 notify.1 [watch unwatch cookie 94217467465088] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.436+0000 7f573b8bb880 20 remove_watcher() i=1 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.436+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:139 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94217467466240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baac00 con 0x55b0b6b81800 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.437+0000 7f573a060640 1 -- 172.21.15.138:0/2571899439 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(139 notify.2 [watch unwatch cookie 94217467466240] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98fc0 con 0x55b0b6b81800 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.437+0000 7f573b8bb880 20 remove_watcher() i=2 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.437+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:140 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94217467467392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6baa400 con 0x55b0b6b81000 2024-04-27T05:22:39.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.438+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(140 notify.3 [watch unwatch cookie 94217467467392] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b99200 con 0x55b0b6b81000 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.438+0000 7f573b8bb880 20 remove_watcher() i=3 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.438+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94217467468544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cea000 con 0x55b0b75ff000 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.439+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.1 v2:172.21.15.138:6800/29864 47 ==== osd_op_reply(141 notify.4 [watch unwatch cookie 94217467468544] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b75ff000 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.439+0000 7f573b8bb880 20 remove_watcher() i=4 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.439+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94217467469696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6cea800 con 0x55b0b6b81c00 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.440+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 57 ==== osd_op_reply(142 notify.5 [watch unwatch cookie 94217467469696] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.440+0000 7f573b8bb880 20 remove_watcher() i=5 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.440+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:143 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94217467470848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ceb400 con 0x55b0b6b81c00 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.441+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 58 ==== osd_op_reply(143 notify.6 [watch unwatch cookie 94217467470848] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.441+0000 7f573b8bb880 20 remove_watcher() i=6 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.441+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:144 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94217467472000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b0b6ceb800 con 0x55b0b6b81c00 2024-04-27T05:22:39.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573a861640 1 -- 172.21.15.138:0/2571899439 <== osd.0 v2:172.21.15.114:6800/27747 59 ==== osd_op_reply(144 notify.7 [watch unwatch cookie 94217467472000] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55b0b6b98900 con 0x55b0b6b81c00 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 20 remove_watcher() i=7 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6b81c00 msgr2=0x55b0b6aeb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6b81c00 0x55b0b6aeb180 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b75ff000 msgr2=0x55b0b6ae9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b75ff000 0x55b0b6ae9b80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6b81000 msgr2=0x55b0b6aeac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6b81000 0x55b0b6aeac00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6b81800 msgr2=0x55b0b6aea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6b81800 0x55b0b6aea100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b6b80000 msgr2=0x55b0b6ae9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b6b80000 0x55b0b6ae9600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55b0b6913380 tx=0x55b0b776ef60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 msgr2=0x55b0b6ae8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.442+0000 7f573b8bb880 1 --2- 172.21.15.138:0/2571899439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b75fec00 0x55b0b6ae8b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55b0b76c2b40 tx=0x55b0b6919b00 comp rx=0 tx=0).stop 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 reap_dead start 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573985f640 1 -- 172.21.15.138:0/2571899439 reap_dead start 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 shutdown_connections 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 >> 172.21.15.138:0/2571899439 conn(0x55b0b6807400 msgr2=0x55b0b6911000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 shutdown_connections 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/2571899439 wait complete. 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6c4c400 msgr2=0x55b0b6c48580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55b0b6c4c400 0x55b0b6c48580 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b7764800 msgr2=0x55b0b6c48000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55b0b7764800 0x55b0b6c48000 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6c4d800 msgr2=0x55b0b6c49080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55b0b6c4d800 0x55b0b6c49080 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6c4cc00 msgr2=0x55b0b6c48b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55b0b6c4cc00 0x55b0b6c48b00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b7764c00 msgr2=0x55b0b6aeb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b7764c00 0x55b0b6aeb700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55b0b77701b0 tx=0x55b0b776fe60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573a060640 1 -- 172.21.15.138:0/155823548 reap_dead start 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 msgr2=0x55b0b6aea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55b0b6b63800 tx=0x55b0b776f080 comp rx=0 tx=0).stop 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 shutdown_connections 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 --2- 172.21.15.138:0/155823548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6b80c00 0x55b0b6aea680 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 >> 172.21.15.138:0/155823548 conn(0x55b0b6bab800 msgr2=0x55b0b6911800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 shutdown_connections 2024-04-27T05:22:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.443+0000 7f573b8bb880 1 -- 172.21.15.138:0/155823548 wait complete. 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b75ffc00 msgr2=0x55b0b6ae9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b75ffc00 0x55b0b6ae9080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55b0b7708180 tx=0x55b0b6919c20 comp rx=0 tx=0).stop 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 msgr2=0x55b0b6ae8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x55b0b76c32c0 tx=0x55b0b6918c60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 shutdown_connections 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55b0b75ffc00 0x55b0b6ae9080 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 --2- 172.21.15.138:0/1228124652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0b6807000 0x55b0b6ae8580 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 >> 172.21.15.138:0/1228124652 conn(0x55b0b6806c00 msgr2=0x55b0b690f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 shutdown_connections 2024-04-27T05:22:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.444+0000 7f573b8bb880 1 -- 172.21.15.138:0/1228124652 wait complete. 2024-04-27T05:22:39.466 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-27T05:22:39.466 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-27T05:22:39.466 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-04-27T05:22:39.535 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:22:39.535 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:22:39.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.534+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.534+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.534+0000 7f4c77c10880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.534+0000 7f4c77c10880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56554b156600 con 0x56554ae7f400 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.534+0000 7f4c77c10880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x56554aebe280 con 0x56554ae7f000 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.535+0000 7f4c74131640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.535+0000 7f4c73930640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x56554ae7f400 0x56554af86800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:48318/0 (socket says 172.21.15.138:48318) 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.535+0000 7f4c73930640 1 -- 172.21.15.138:0/2099677232 learned_addr learned my addr 172.21.15.138:0/2099677232 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.535+0000 7f4c74131640 1 -- 172.21.15.138:0/2099677232 >> v1:172.21.15.114:6789/0 conn(0x56554ae7f400 legacy=0x56554af86800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.535+0000 7f4c74131640 1 -- 172.21.15.138:0/2099677232 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554ae83380 con 0x56554ae7f000 2024-04-27T05:22:39.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c74131640 1 --2- 172.21.15.138:0/2099677232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x56554af8ad50 tx=0x56554af90de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e40135d489c56d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c7312f640 1 -- 172.21.15.138:0/2099677232 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae97ce0 con 0x56554ae7f000 2024-04-27T05:22:39.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c7312f640 1 -- 172.21.15.138:0/2099677232 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554b157b00 con 0x56554ae7f000 2024-04-27T05:22:39.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c7312f640 1 -- 172.21.15.138:0/2099677232 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae97e40 con 0x56554ae7f000 2024-04-27T05:22:39.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c77c10880 1 -- 172.21.15.138:0/2099677232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 msgr2=0x56554b160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c77c10880 1 --2- 172.21.15.138:0/2099677232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x56554af8ad50 tx=0x56554af90de0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c77c10880 1 -- 172.21.15.138:0/2099677232 shutdown_connections 2024-04-27T05:22:39.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c77c10880 1 --2- 172.21.15.138:0/2099677232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.537+0000 7f4c77c10880 1 -- 172.21.15.138:0/2099677232 >> 172.21.15.138:0/2099677232 conn(0x56554ae7ec00 msgr2=0x56554af86000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.538+0000 7f4c77c10880 1 -- 172.21.15.138:0/2099677232 shutdown_connections 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.538+0000 7f4c77c10880 1 -- 172.21.15.138:0/2099677232 wait complete. 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.538+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.538+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.538+0000 7f4c77c10880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.538+0000 7f4c77c10880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554aebe780 con 0x56554ae7f000 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c74131640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c74131640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52508/0 (socket says 172.21.15.138:52508) 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c74131640 1 -- 172.21.15.138:0/2269580334 learned_addr learned my addr 172.21.15.138:0/2269580334 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c74131640 1 -- 172.21.15.138:0/2269580334 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554ae82ea0 con 0x56554ae7f000 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c74131640 1 --2- 172.21.15.138:0/2269580334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x56554bc83560 tx=0x56554af90cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acc128fac7f77f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c7292e640 1 -- 172.21.15.138:0/2269580334 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae974a0 con 0x56554ae7f000 2024-04-27T05:22:39.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c7292e640 1 -- 172.21.15.138:0/2269580334 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554b157e00 con 0x56554ae7f000 2024-04-27T05:22:39.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c7292e640 1 -- 172.21.15.138:0/2269580334 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae97ce0 con 0x56554ae7f000 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 -- 172.21.15.138:0/2269580334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 msgr2=0x56554b160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 --2- 172.21.15.138:0/2269580334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x56554bc83560 tx=0x56554af90cc0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 -- 172.21.15.138:0/2269580334 shutdown_connections 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 --2- 172.21.15.138:0/2269580334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 -- 172.21.15.138:0/2269580334 >> 172.21.15.138:0/2269580334 conn(0x56554ae7ec00 msgr2=0x56554af87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 -- 172.21.15.138:0/2269580334 shutdown_connections 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.539+0000 7f4c77c10880 1 -- 172.21.15.138:0/2269580334 wait complete. 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c77c10880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c77c10880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554aebe8c0 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c74131640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c74131640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52518/0 (socket says 172.21.15.138:52518) 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c74131640 1 -- 172.21.15.138:0/4264793420 learned_addr learned my addr 172.21.15.138:0/4264793420 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c74131640 1 -- 172.21.15.138:0/4264793420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554ae836c0 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.540+0000 7f4c74131640 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x56554bd3ade0 tx=0x56554af90c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c7112b640 1 -- 172.21.15.138:0/4264793420 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae97e40 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c7112b640 1 -- 172.21.15.138:0/4264793420 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554bc5e000 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56554ae831e0 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c7112b640 1 -- 172.21.15.138:0/4264793420 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae974a0 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56554aef0ea0 con 0x56554ae7f000 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c77c10880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c77c10880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554aea2280 con 0x56554bc76c00 2024-04-27T05:22:39.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c7112b640 1 -- 172.21.15.138:0/4264793420 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56554aea8b00 con 0x56554ae7f000 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c74932640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c7112b640 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bc77c00 0x56554b161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c74932640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52524/0 (socket says 172.21.15.138:52524) 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c74932640 1 -- 172.21.15.138:0/4134164599 learned_addr learned my addr 172.21.15.138:0/4134164599 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.541+0000 7f4c7112b640 1 -- 172.21.15.138:0/4264793420 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x56554ae91a40 con 0x56554ae7f000 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c74131640 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bc77c00 0x56554b161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c74932640 1 -- 172.21.15.138:0/4134164599 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554aef0d00 con 0x56554bc76c00 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c74932640 1 --2- 172.21.15.138:0/4134164599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x56554bd7f5c0 tx=0x56554af91f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bccbadaecccc488 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c74131640 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bc77c00 0x56554b161080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x56554bd7e2a0 tx=0x56554af91c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c6e926640 1 -- 172.21.15.138:0/4134164599 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae97e40 con 0x56554bc76c00 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c6e926640 1 -- 172.21.15.138:0/4134164599 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554bc5e600 con 0x56554bc76c00 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c6e926640 1 -- 172.21.15.138:0/4134164599 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae974a0 con 0x56554bc76c00 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 -- 172.21.15.138:0/4134164599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 msgr2=0x56554b160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 --2- 172.21.15.138:0/4134164599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x56554bd7f5c0 tx=0x56554af91f80 comp rx=0 tx=0).stop 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 -- 172.21.15.138:0/4134164599 shutdown_connections 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 --2- 172.21.15.138:0/4134164599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 -- 172.21.15.138:0/4134164599 >> 172.21.15.138:0/4134164599 conn(0x56554ae7f400 msgr2=0x56554af89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 -- 172.21.15.138:0/4134164599 shutdown_connections 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.542+0000 7f4c77c10880 1 -- 172.21.15.138:0/4134164599 wait complete. 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c77c10880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c77c10880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554aea3180 con 0x56554bc76c00 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c74932640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c74932640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52528/0 (socket says 172.21.15.138:52528) 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 learned_addr learned my addr 172.21.15.138:0/3366995325 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554aef0b60 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.543+0000 7f4c74932640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x56554bd3ae40 tx=0x56554af91a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c6d123640 1 -- 172.21.15.138:0/3366995325 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae97e40 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c6d123640 1 -- 172.21.15.138:0/3366995325 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554bc5e780 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56554aef09c0 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c6d123640 1 -- 172.21.15.138:0/3366995325 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554ae974a0 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56554aef0820 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c6d123640 1 -- 172.21.15.138:0/3366995325 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56554aea9600 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c6d123640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554b200000 0x56554b161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.544+0000 7f4c6d123640 1 -- 172.21.15.138:0/3366995325 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x56554ae91c00 con 0x56554bc76c00 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bc77000 0x56554b161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c73930640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554b200000 0x56554b161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554bc77000 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c74932640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bc77000 0x56554b161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c73930640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554b200000 0x56554b161600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56554bdecf60 tx=0x56554bdeeea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c74932640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bc77000 0x56554b161b80 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 20 realm 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77800 con 0x56554bc77000 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.545+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201000 con 0x56554bc77000 2024-04-27T05:22:39.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201400 con 0x56554bc77000 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b201800 0x56554b162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201c00 con 0x56554b201800 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c74131640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b201800 0x56554b162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c74131640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b201800 0x56554b162100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.546+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c77c10880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c77c10880 20 searching for the correct realm 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201000 con 0x56554b201800 2024-04-27T05:22:39.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c74131640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b201000 0x56554b162c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201c00 con 0x56554b201000 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c73930640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b201000 0x56554b162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.547+0000 7f4c73930640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b201000 0x56554b162c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c73930640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b201c00 0x56554b163180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c74932640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b201c00 0x56554b163180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c74932640 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b201c00 0x56554b163180 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.548+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201000 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.549+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.550+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201000 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.551+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.551+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.551+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.551+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.551+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.551+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.552+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.553+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201000 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.554+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201800 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201c00 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554bc77000 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b200c00 con 0x56554b201000 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c77c10880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c77c10880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c77c10880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c77c10880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.555+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201400 con 0x56554bc77000 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554bc77000 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:22:39.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77800 con 0x56554b201800 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 zone default found 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 4 Realm: () 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 zonegroup default 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226000 con 0x56554b201c00 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.556+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226400 con 0x56554b201800 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226800 con 0x56554b201800 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 20 started sync module instance, tier type = 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:22:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.557+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226c00 con 0x56554b201000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.558+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.558+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94924331911424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227000 con 0x56554b201000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.558+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227400 con 0x56554bc77000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.559+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94924331911424] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.559+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.559+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94924331912576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227800 con 0x56554bc77000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.559+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227c00 con 0x56554b201800 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.562+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94924331912576] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.562+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.562+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94924331913728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226400 con 0x56554b201800 2024-04-27T05:22:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.562+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226800 con 0x56554b201000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.563+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94924331913728] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.563+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.563+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94924331914880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227c00 con 0x56554b201000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.563+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226000 con 0x56554bc77000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.564+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94924331914880] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.565+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.565+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94924331916032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77800 con 0x56554bc77000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.565+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554b201c00 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.566+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.566+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94924331916032] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.566+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94924331917184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b201400 con 0x56554b201c00 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.566+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bddd800 con 0x56554b201c00 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.567+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94924331917184] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.568+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.568+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94924331918336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bddd400 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.568+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226c00 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94924331918336] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94924331919488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227000 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=0 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=1 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=2 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=3 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=4 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=5 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.569+0000 7f4c77c10880 20 add_watcher() i=6 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.570+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94924331919488] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554b201c00 2024-04-27T05:22:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.570+0000 7f4c77c10880 20 add_watcher() i=7 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.570+0000 7f4c77c10880 2 all 8 watchers are set, enabling cache 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.570+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226800 con 0x56554bc77000 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.570+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.570+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94924331920640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227400 con 0x56554bc77000 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94924331920640] v31'229 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56554b212900 con 0x56554bc77000 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c5b0ff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c77c10880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c77c10880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c77c10880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.571+0000 7f4c77c10880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554aea2780 con 0x56554b200c00 2024-04-27T05:22:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c73930640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c73930640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52534/0 (socket says 172.21.15.138:52534) 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c73930640 1 -- 172.21.15.138:0/2333366321 learned_addr learned my addr 172.21.15.138:0/2333366321 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c73930640 1 -- 172.21.15.138:0/2333366321 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554aef0680 con 0x56554b200c00 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c73930640 1 --2- 172.21.15.138:0/2333366321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x56554b1de930 tx=0x56554bdeee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ca015b4916f480 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c5a8fe640 1 -- 172.21.15.138:0/2333366321 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554b228580 con 0x56554b200c00 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c5a8fe640 1 -- 172.21.15.138:0/2333366321 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554bc5e000 con 0x56554b200c00 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.572+0000 7f4c5a8fe640 1 -- 172.21.15.138:0/2333366321 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554b2286e0 con 0x56554b200c00 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- 172.21.15.138:0/2333366321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 msgr2=0x56554b162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 --2- 172.21.15.138:0/2333366321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x56554b1de930 tx=0x56554bdeee40 comp rx=0 tx=0).stop 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- 172.21.15.138:0/2333366321 shutdown_connections 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 --2- 172.21.15.138:0/2333366321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- 172.21.15.138:0/2333366321 >> 172.21.15.138:0/2333366321 conn(0x56554b227800 msgr2=0x56554af89800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- 172.21.15.138:0/2333366321 shutdown_connections 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- 172.21.15.138:0/2333366321 wait complete. 2024-04-27T05:22:39.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 Processor -- start 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- start start 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.573+0000 7f4c77c10880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x56554aea2500 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c73930640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c73930640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:52550/0 (socket says 172.21.15.138:52550) 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 learned_addr learned my addr 172.21.15.138:0/429002414 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56554aef04e0 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c73930640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x56554b1df860 tx=0x56554bdeefc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c590fb640 1 -- 172.21.15.138:0/429002414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554b228840 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c590fb640 1 -- 172.21.15.138:0/429002414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56554bc5ea80 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56554ae83380 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c590fb640 1 -- 172.21.15.138:0/429002414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56554b228580 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.574+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56554ae82ea0 con 0x56554b200c00 2024-04-27T05:22:39.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c590fb640 1 -- 172.21.15.138:0/429002414 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56554aea9b80 con 0x56554b200c00 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c590fb640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bddd800 0x56554b163700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c590fb640 1 -- 172.21.15.138:0/429002414 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x56554ae91dc0 con 0x56554b200c00 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c74932640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bddd800 0x56554b163700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bddd400 0x56554b2c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2ca000 con 0x56554bddd400 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c74131640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bddd400 0x56554b2c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c74932640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bddd800 0x56554b163700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x56554b1df230 tx=0x56554bdefda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.575+0000 7f4c74131640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bddd400 0x56554b2c6000 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.576+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'61 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554bddd400 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.576+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b2ca400 0x56554b2c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.576+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2ca800 con 0x56554b2ca400 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.577+0000 7f4c73930640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b2ca400 0x56554b2c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.577+0000 7f4c73930640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b2ca400 0x56554b2c6580 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.578+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.578+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b2cac00 0x56554b2c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.578+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb000 con 0x56554b2cac00 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.578+0000 7f4c74932640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b2cac00 0x56554b2c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.578+0000 7f4c74932640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b2cac00 0x56554b2c6b00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.579+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.579+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb400 con 0x56554bddd400 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.580+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'230 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554bddd400 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.580+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b2cb800 0x56554b2c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.580+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cbc00 con 0x56554b2cb800 2024-04-27T05:22:39.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.580+0000 7f4c74131640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b2cb800 0x56554b2c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:22:39.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.580+0000 7f4c74131640 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b2cb800 0x56554b2c7080 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:22:39.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.581+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'77 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.581+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2000 con 0x56554b2cac00 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.582+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'62 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.582+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554bddd400 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.583+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'36 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554bddd400 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.583+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2800 con 0x56554b2ca400 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.584+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'53 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.584+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2c00 con 0x56554b2cb800 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.585+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.585+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3000 con 0x56554b2cac00 2024-04-27T05:22:39.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.586+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.586+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3400 con 0x56554b2cb800 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.587+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'78 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.587+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3800 con 0x56554b2ca400 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.588+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'34 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.588+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3c00 con 0x56554b2cb800 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.589+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.589+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554b2ca400 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.590+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'70 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.590+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2c00 con 0x56554bddd400 2024-04-27T05:22:39.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.591+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'231 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554bddd400 2024-04-27T05:22:39.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.591+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3400 con 0x56554b2cac00 2024-04-27T05:22:39.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.592+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'77 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.592+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2ca000 con 0x56554b2cb800 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.593+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'79 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.593+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb400 con 0x56554b2cac00 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.594+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'39 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.594+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cbc00 con 0x56554bddd400 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.595+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554bddd400 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.595+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226400 con 0x56554bddd400 2024-04-27T05:22:39.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.596+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'232 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554bddd400 2024-04-27T05:22:39.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.596+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2000 con 0x56554b2cac00 2024-04-27T05:22:39.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.597+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'78 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.597+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3000 con 0x56554b2cb800 2024-04-27T05:22:39.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.598+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'43 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.598+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3400 con 0x56554b2cac00 2024-04-27T05:22:39.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.598+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.598+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2800 con 0x56554b2cac00 2024-04-27T05:22:39.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.599+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'45 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.599+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3800 con 0x56554b2cac00 2024-04-27T05:22:39.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.600+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.600+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554b2cac00 2024-04-27T05:22:39.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.601+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'63 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.601+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb000 con 0x56554b2ca400 2024-04-27T05:22:39.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.602+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'36 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.602+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb400 con 0x56554b2cac00 2024-04-27T05:22:39.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.603+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'39 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.603+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2ca800 con 0x56554b2ca400 2024-04-27T05:22:39.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.604+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'54 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.604+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554b2cac00 2024-04-27T05:22:39.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.605+0000 7f4c74932640 1 -- 172.21.15.138:0/429002414 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'79 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b2cac00 2024-04-27T05:22:39.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.605+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226c00 con 0x56554b2ca400 2024-04-27T05:22:39.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.606+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'71 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.607+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b227000 con 0x56554b2cb800 2024-04-27T05:22:39.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.607+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b2cb800 2024-04-27T05:22:39.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c568f6640 20 reqs_thread_entry: start 2024-04-27T05:22:39.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c77c10880 20 init_complete bucket index max shards: 11 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c77c10880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c548f2640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:10 2024 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c540d0640 20 reqs_thread_entry: start 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c77c10880 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.608+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348000 con 0x56554b201c00 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 20 rados_obj.operate() r=0 bl.length=346 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348400 con 0x56554b201c00 2024-04-27T05:22:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(63 fud.buckets [call out=184b] v0'0 uv5 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 : miss 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.7 8:e041234b:root::.bucket.meta.myfoo7%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.8:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348800 con 0x56554b201c00 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.609+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(64 .bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 20 rados_obj.operate() r=0 bl.length=353 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 info.flags=0x17 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 to cache LRU end 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 4.27 4:e5cd4499:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348c00 con 0x56554bc77000 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(65 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 4.13 4:c8dc6cb7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349000 con 0x56554b201000 2024-04-27T05:22:39.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 4.3f 4:ff97fb24:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349400 con 0x56554b201000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.610+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 4.24 4:2762930c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349800 con 0x56554b201c00 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 4.7 4:e34f789a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349c00 con 0x56554b201c00 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 4.9 4:929cabb3:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348000 con 0x56554bc77000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 [call out=377b] v0'0 uv16 ondisk = 0) v8 ==== 194+0+377 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c77c10880 10 cls_bucket_list_unordered: got fourteen 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.611+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:71 4.c 4:33672c0c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554bc77000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.612+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.612+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.31 4:8d6b570d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb400 con 0x56554bc77000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.612+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.612+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:73 4.c 4:33f25152:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2000 con 0x56554bc77000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.612+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8 [call out=85b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.612+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:74 4.2f 4:f69c3374:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3400 con 0x56554bc77000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9 [call out=85b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:75 4.1d 4:b86e2516:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2800 con 0x56554b201000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 20 get_obj_state: rctx=0x56554b2bc070 obj=myfoo7:fourteen state=0x56554b34e1e8 s->prefetch_data=0 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:76 3.19 3:989dee87:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.8_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3800 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(76 2f19b8ff-2608-4d86-8f1c-430623205476.4231.8_fourteen [getxattrs out=854b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+870 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 10 manifest: total_size = 8 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.2228634731808716179 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 20 get_obj_state: rctx=0x56554b2bc070 obj=myfoo7:fourteen state=0x56554b34e1e8 s->prefetch_data=0 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 20 prepare_atomic_modification: state is not atomic. state=0x56554b34e1e8 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.613+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:77 4.9 4:929cabb3:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.615+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(77 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 [stat,call,call] v31'20 uv20 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.615+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:78 3.19 3:989dee87:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.8_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2000 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.615+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(78 2f19b8ff-2608-4d86-8f1c-430623205476.4231.8_fourteen [call] v31'2 uv2 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:79 4.9 4:929cabb3:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554bc77000 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 20 list_objects_ordered: starting attempt 1 2024-04-27T05:22:39.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:80 4.27 4:e5cd4499:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348400 con 0x56554bc77000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 4.13 4:c8dc6cb7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348800 con 0x56554b201000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:82 4.3f 4:ff97fb24:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348c00 con 0x56554b201000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 4.24 4:2762930c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349800 con 0x56554b201c00 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:84 4.7 4:e34f789a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349c00 con 0x56554b201c00 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:85 4.9 4:929cabb3:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348000 con 0x56554bc77000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(80 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:86 4.c 4:33672c0c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349000 con 0x56554bc77000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(81 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:87 4.31 4:8d6b570d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349400 con 0x56554bc77000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(82 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(83 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:88 4.c 4:33f25152:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348800 con 0x56554bc77000 2024-04-27T05:22:39.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:89 4.2f 4:f69c3374:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2800 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:90 4.1d 4:b86e2516:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226c00 con 0x56554b201000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== osd_op_reply(84 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(90 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(87 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.616+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(86 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(89 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9 [call out=85b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(79 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 [stat,call,call] v31'21 uv21 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c6e926640 20 handle_completion(): completion ok for obj=fourteen 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(88 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8 [call out=85b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 36 ==== osd_op_reply(85 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 [call out=124b] v0'0 uv21 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T05:22:39.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[2f19b8ff-2608-4d86-8f1c-430623205476.4231.8])"; uploads.size()=0; is_truncated=0 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo7[2f19b8ff-2608-4d86-8f1c-430623205476.4231.8]) 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:91 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb000 con 0x56554b201000 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(91 myfoo7 [call out=48b,stat out=16b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv42 ondisk = 0) v8 ==== 276+0+305 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x17 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 lifecycle: adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.617+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:92 8.7 8:e041234b:root::.bucket.meta.myfoo7%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.8:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348400 con 0x56554b201c00 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(92 .bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 [delete,create,call,call,writefull 0~353,setxattr (123)] v31'3 uv3 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c77c10880 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 info.flags=0x17 2024-04-27T05:22:39.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c77c10880 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 to cache LRU end 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c77c10880 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c77c10880 10 lifecycle: distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8, ofs0, ns] 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:93 7.3 7:c609908c:::notify.5:head [notify cookie 94924331924096 in=739b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349c00 con 0x56554b201c00 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== watch-notify(notify (1) cookie 94924331917184 notify 133143986214 ret 0) v3 ==== 769+0+0 (crc 0 0 0) 0x56554b27fa00 con 0x56554b201c00 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c6e125640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986214 cookie 94924331917184 notifier 4837 bl.length()=727 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c6e125640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 info.flags=0x17 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c6e125640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 to cache LRU end 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== osd_op_reply(93 notify.5 [notify cookie 94924331924096 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c6e125640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c6e125640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:94 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a000 con 0x56554b201c00 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.619+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(94 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.620+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== watch-notify(notify_complete (2) cookie 94924331924096 notify 133143986214 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56554b27fba0 con 0x56554b201c00 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.620+0000 7f4c77c10880 20 lifecycle: handle_bi_update(): bucket=:myfoo7[2f19b8ff-2608-4d86-8f1c-430623205476.4231.8]): orig_sources= new_sources= 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.620+0000 7f4c77c10880 20 lifecycle: handle_bi_update(): bucket=:myfoo7[2f19b8ff-2608-4d86-8f1c-430623205476.4231.8]): potential sources added=[] removed=[] 2024-04-27T05:22:39.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.620+0000 7f4c77c10880 20 lifecycle: handle_bi_update(): bucket=:myfoo7[2f19b8ff-2608-4d86-8f1c-430623205476.4231.8]): orig_dests= new_dests= 2024-04-27T05:22:39.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.620+0000 7f4c77c10880 20 lifecycle: handle_bi_update(): bucket=:myfoo7[2f19b8ff-2608-4d86-8f1c-430623205476.4231.8]): potential dests added=[] removed=[] 2024-04-27T05:22:39.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.620+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:33 6.0 6:05588ce8:lc::lc.6:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348000 con 0x56554b2ca400 2024-04-27T05:22:39.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.621+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(33 lc.6 [call] v31'72 uv72 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.621+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:34 6.0 6:05588ce8:lc::lc.6:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349000 con 0x56554b2ca400 2024-04-27T05:22:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.621+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(34 lc.6 [call] v31'73 uv73 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.621+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 6.0 6:05588ce8:lc::lc.6:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554b2ca400 2024-04-27T05:22:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c73930640 1 -- 172.21.15.138:0/429002414 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(35 lc.6 [call] v31'74 uv74 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b2ca400 2024-04-27T05:22:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:95 4.27 4:e5cd4499:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3400 con 0x56554bc77000 2024-04-27T05:22:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:96 4.13 4:c8dc6cb7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3800 con 0x56554b201000 2024-04-27T05:22:39.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:97 4.3f 4:ff97fb24:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554b201000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:98 4.24 4:2762930c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2000 con 0x56554b201c00 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:99 4.7 4:e34f789a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a400 con 0x56554b201c00 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:100 4.9 4:929cabb3:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a800 con 0x56554bc77000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.622+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:101 4.c 4:33672c0c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35ac00 con 0x56554bc77000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:102 4.31 4:8d6b570d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b000 con 0x56554bc77000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:103 4.c 4:33f25152:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b400 con 0x56554bc77000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:104 4.2f 4:f69c3374:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b800 con 0x56554bc77000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:105 4.1d 4:b86e2516:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35bc00 con 0x56554b201000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(98 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== osd_op_reply(99 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(97 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(96 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 37 ==== osd_op_reply(95 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 38 ==== osd_op_reply(100 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 [call out=124b] v0'0 uv21 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 39 ==== osd_op_reply(101 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(105 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 40 ==== osd_op_reply(102 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 41 ==== osd_op_reply(104 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9 [call out=85b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 42 ==== osd_op_reply(103 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8 [call out=85b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.623+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:106 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2800 con 0x56554b201c00 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.624+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== osd_op_reply(106 fud.buckets [call] v31'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.624+0000 7f4c77c10880 10 cache get: name=default.rgw.meta+root+myfoo7 : hit (requested=0x11, cached=0x17) 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.624+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:107 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3400 con 0x56554b201000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(107 myfoo7 [call,call,delete] v31'43 uv43 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c77c10880 10 removing default.rgw.meta+root+myfoo7 from cache 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c77c10880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 94924331924096 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2000 con 0x56554b201000 2024-04-27T05:22:39.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== watch-notify(notify (1) cookie 94924331914880 notify 133143986195 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x56554b27fd40 con 0x56554b201000 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c6e125640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986195 cookie 94924331914880 notifier 4837 bl.length()=150 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(108 notify.3 [notify cookie 94924331924096 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.625+0000 7f4c6e125640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348000 con 0x56554b201000 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.626+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 24 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.626+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 25 ==== watch-notify(notify_complete (2) cookie 94924331924096 notify 133143986195 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56554b27b860 con 0x56554b201000 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.626+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:110 8.7 8:e041234b:root::.bucket.meta.myfoo7%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.8:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3800 con 0x56554b201c00 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== osd_op_reply(110 .bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 [call,call,delete] v31'4 uv4 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 from cache 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:2f19b8ff-2608-4d86-8f1c-430623205476.4231.8, ofs0, ns] 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [notify cookie 94924331924096 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554b201c00 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 33 ==== watch-notify(notify (1) cookie 94924331917184 notify 133143986215 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x56554b27fba0 con 0x56554b201c00 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c6e125640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986215 cookie 94924331917184 notifier 4837 bl.length()=207 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 34 ==== osd_op_reply(111 notify.5 [notify cookie 94924331924096 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c6e125640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:112 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348400 con 0x56554b201c00 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 35 ==== osd_op_reply(112 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 36 ==== watch-notify(notify_complete (2) cookie 94924331924096 notify 133143986215 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56554b27e1a0 con 0x56554b201c00 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:113 4.27 4:e5cd4499:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554bc77000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:114 4.13 4:c8dc6cb7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2ca800 con 0x56554b201000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:115 4.3f 4:ff97fb24:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cb000 con 0x56554b201000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:116 4.24 4:2762930c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226c00 con 0x56554b201c00 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:117 4.7 4:e34f789a:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a000 con 0x56554b201c00 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.627+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:118 4.9 4:929cabb3:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a400 con 0x56554bc77000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.628+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:119 4.c 4:33672c0c:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348400 con 0x56554bc77000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.628+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:120 4.31 4:8d6b570d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348800 con 0x56554bc77000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.628+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:121 4.c 4:33f25152:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b348c00 con 0x56554bc77000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.628+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:122 4.2f 4:f69c3374:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349000 con 0x56554bc77000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.628+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:123 4.1d 4:b86e2516:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349400 con 0x56554b201000 2024-04-27T05:22:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.629+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 26 ==== osd_op_reply(114 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.1 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.629+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 37 ==== osd_op_reply(116 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.3 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.629+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 27 ==== osd_op_reply(115 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.2 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.629+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 43 ==== osd_op_reply(113 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.0 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.629+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 44 ==== osd_op_reply(118 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.5 [delete] v31'22 uv22 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.630+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 45 ==== osd_op_reply(122 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.9 [delete] v31'11 uv11 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.630+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 46 ==== osd_op_reply(120 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.7 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.630+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 47 ==== osd_op_reply(119 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.6 [delete] v31'11 uv11 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.630+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 38 ==== osd_op_reply(117 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.4 [delete] v31'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.630+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 28 ==== osd_op_reply(123 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.10 [delete] v31'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 48 ==== osd_op_reply(121 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.8.8 [delete] v31'12 uv12 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 : miss 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 20 rados->read ofs=0 len=0 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:124 6.18 6:1b7107a2:::pubsub..bucket.myfoo7%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.8:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349800 con 0x56554b201c00 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 39 ==== osd_op_reply(124 pubsub..bucket.myfoo7/2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:22:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 info.flags=0x0 2024-04-27T05:22:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 10 adding default.rgw.log++pubsub..bucket.myfoo7/2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 to cache LRU end 2024-04-27T05:22:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.632+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:125 6.18 6:1b7107a2:::pubsub..bucket.myfoo7%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b349c00 con 0x56554b201c00 2024-04-27T05:22:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.633+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 40 ==== osd_op_reply(125 pubsub..bucket.myfoo7/2f19b8ff-2608-4d86-8f1c-430623205476.4231.8 [delete] v31'36 uv29 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.633+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:126 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a800 con 0x56554b201c00 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.634+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 41 ==== osd_op_reply(126 fud.buckets [call] v31'7 uv7 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.634+0000 7f4c77c10880 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.634+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:127 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35ac00 con 0x56554b201c00 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 42 ==== osd_op_reply(127 p5YnriCv1nAtykxBrupQ [delete] v31'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c77c10880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:128 7.15 7:a93a5511:::notify.2:head [notify cookie 94924331924096 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b000 con 0x56554b201800 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94924331913728 notify 133143986196 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x56554bd84d00 con 0x56554b201800 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c6e125640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986196 cookie 94924331913728 notifier 4837 bl.length()=170 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(128 notify.2 [notify cookie 94924331924096 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.635+0000 7f4c6e125640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:129 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3c00 con 0x56554b201800 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.636+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== watch-notify(notify_complete (2) cookie 94924331924096 notify 133143986196 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56554bd84ea0 con 0x56554b201800 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.636+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(129 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.636+0000 7f4c77c10880 10 removing email index: 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.636+0000 7f4c77c10880 10 removing user buckets index 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.636+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:130 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b400 con 0x56554b201c00 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 43 ==== osd_op_reply(130 fud.buckets [delete] v31'8 uv8 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c77c10880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:131 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94924331924096 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b800 con 0x56554b201c00 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 44 ==== watch-notify(notify (1) cookie 94924331918336 notify 133143986216 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x56554b27e1a0 con 0x56554b201c00 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c6e125640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986216 cookie 94924331918336 notifier 4837 bl.length()=160 2024-04-27T05:22:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 45 ==== osd_op_reply(131 notify.6 [notify cookie 94924331924096 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c6e125640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:132 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2ca000 con 0x56554b201c00 2024-04-27T05:22:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 46 ==== osd_op_reply(132 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 47 ==== watch-notify(notify_complete (2) cookie 94924331924096 notify 133143986216 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56554b27e340 con 0x56554b201c00 2024-04-27T05:22:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c77c10880 10 removing user index: fud 2024-04-27T05:22:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.637+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:133 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a000 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.638+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 48 ==== osd_op_reply(133 fud [call,call,delete] v31'7 uv7 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.638+0000 7f4c77c10880 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.638+0000 7f4c77c10880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.638+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94924331924096 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b226c00 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 49 ==== watch-notify(notify (1) cookie 94924331917184 notify 133143986217 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x56554b27e340 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c6e125640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986217 cookie 94924331917184 notifier 4837 bl.length()=152 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 50 ==== osd_op_reply(134 notify.5 [notify cookie 94924331924096 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c6e125640 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2cbc00 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 51 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 52 ==== watch-notify(notify_complete (2) cookie 94924331924096 notify 133143986217 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56554b27e4e0 con 0x56554b201c00 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c5b0ff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.639+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:136 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94924331920640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554bc77400 con 0x56554bc77000 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.640+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 49 ==== osd_op_reply(136 data_loggenerations_metadata [watch unwatch cookie 94924331920640] v31'233 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.640+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:137 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94924331911424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2800 con 0x56554b201000 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.641+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 29 ==== osd_op_reply(137 notify.0 [watch unwatch cookie 94924331911424] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.641+0000 7f4c77c10880 20 remove_watcher() i=0 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.641+0000 7f4c77c10880 2 removed watcher, disabling cache 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.641+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:138 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94924331912576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f3800 con 0x56554bc77000 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.642+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 50 ==== osd_op_reply(138 notify.1 [watch unwatch cookie 94924331912576] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.642+0000 7f4c77c10880 20 remove_watcher() i=1 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.642+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:139 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94924331913728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b2f2400 con 0x56554b201800 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.643+0000 7f4c74131640 1 -- 172.21.15.138:0/3366995325 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(139 notify.2 [watch unwatch cookie 94924331913728] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212fc0 con 0x56554b201800 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.643+0000 7f4c77c10880 20 remove_watcher() i=2 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.643+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:140 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94924331914880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a400 con 0x56554b201000 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.644+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 <== osd.2 v2:172.21.15.138:6809/29865 30 ==== osd_op_reply(140 notify.3 [watch unwatch cookie 94924331914880] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b213200 con 0x56554b201000 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.644+0000 7f4c77c10880 20 remove_watcher() i=3 2024-04-27T05:22:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.644+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94924331916032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35a800 con 0x56554bc77000 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.645+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.1 v2:172.21.15.138:6800/29864 51 ==== osd_op_reply(141 notify.4 [watch unwatch cookie 94924331916032] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554bc77000 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.645+0000 7f4c77c10880 20 remove_watcher() i=4 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.645+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94924331917184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35ac00 con 0x56554b201c00 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.646+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 53 ==== osd_op_reply(142 notify.5 [watch unwatch cookie 94924331917184] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.646+0000 7f4c77c10880 20 remove_watcher() i=5 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.646+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:143 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94924331918336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b400 con 0x56554b201c00 2024-04-27T05:22:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.647+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 54 ==== osd_op_reply(143 notify.6 [watch unwatch cookie 94924331918336] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.647+0000 7f4c77c10880 20 remove_watcher() i=6 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.647+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:144 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94924331919488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56554b35b800 con 0x56554b201c00 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c74932640 1 -- 172.21.15.138:0/3366995325 <== osd.0 v2:172.21.15.114:6800/27747 55 ==== osd_op_reply(144 notify.7 [watch unwatch cookie 94924331919488] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56554b212b40 con 0x56554b201c00 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 20 remove_watcher() i=7 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b201c00 msgr2=0x56554b163180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b201c00 0x56554b163180 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bc77000 msgr2=0x56554b161b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bc77000 0x56554b161b80 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b201000 msgr2=0x56554b162c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b201000 0x56554b162c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b201800 msgr2=0x56554b162100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.648+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b201800 0x56554b162100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554b200000 msgr2=0x56554b161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554b200000 0x56554b161600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56554bdecf60 tx=0x56554bdeeea0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 msgr2=0x56554b160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/3366995325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554bc76c00 0x56554b160b00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x56554bd3ae40 tx=0x56554af91a40 comp rx=0 tx=0).stop 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 reap_dead start 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c73930640 1 -- 172.21.15.138:0/3366995325 reap_dead start 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 shutdown_connections 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 >> 172.21.15.138:0/3366995325 conn(0x56554ae7f400 msgr2=0x56554af89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 shutdown_connections 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/3366995325 wait complete. 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b2ca400 msgr2=0x56554b2c6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x56554b2ca400 0x56554b2c6580 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bddd400 msgr2=0x56554b2c6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x56554bddd400 0x56554b2c6000 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b2cb800 msgr2=0x56554b2c7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x56554b2cb800 0x56554b2c7080 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b2cac00 msgr2=0x56554b2c6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x56554b2cac00 0x56554b2c6b00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bddd800 msgr2=0x56554b163700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bddd800 0x56554b163700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x56554b1df230 tx=0x56554bdefda0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 msgr2=0x56554b162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x56554b1df860 tx=0x56554bdeefc0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c74131640 1 -- 172.21.15.138:0/429002414 reap_dead start 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 shutdown_connections 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 --2- 172.21.15.138:0/429002414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554b200c00 0x56554b162680 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 >> 172.21.15.138:0/429002414 conn(0x56554b227800 msgr2=0x56554af89800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 shutdown_connections 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.649+0000 7f4c77c10880 1 -- 172.21.15.138:0/429002414 wait complete. 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bc77c00 msgr2=0x56554b161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bc77c00 0x56554b161080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x56554bd7e2a0 tx=0x56554af91c20 comp rx=0 tx=0).stop 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 msgr2=0x56554b160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x56554bd3ade0 tx=0x56554af90c60 comp rx=0 tx=0).stop 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 shutdown_connections 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x56554bc77c00 0x56554b161080 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 --2- 172.21.15.138:0/4264793420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56554ae7f000 0x56554b160580 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 >> 172.21.15.138:0/4264793420 conn(0x56554ae7ec00 msgr2=0x56554af87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 shutdown_connections 2024-04-27T05:22:39.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:22:39.650+0000 7f4c77c10880 1 -- 172.21.15.138:0/4264793420 wait complete. 2024-04-27T05:23:19.663 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-04-27T05:23:19.664 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-04-27T05:23:19.664 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-04-27T05:23:19.715 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:19.715 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.713+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.713+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.713+0000 7f5937e30880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c3d81f3400 0x55c3d84d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.713+0000 7f5937e30880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c3d84ca600 con 0x55c3d81f3000 2024-04-27T05:23:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.713+0000 7f5937e30880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c3d8232280 con 0x55c3d81f3400 2024-04-27T05:23:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.714+0000 7f593378f640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55c3d81f3400 0x55c3d84d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.714+0000 7f5933f90640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55c3d81f3000 0x55c3d82fa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38238/0 (socket says 172.21.15.138:38238) 2024-04-27T05:23:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.714+0000 7f5933f90640 1 -- 172.21.15.138:0/3691833226 learned_addr learned my addr 172.21.15.138:0/3691833226 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:23:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.714+0000 7f593378f640 1 -- 172.21.15.138:0/3691833226 >> v1:172.21.15.114:6789/0 conn(0x55c3d81f3000 legacy=0x55c3d82fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.714+0000 7f593378f640 1 -- 172.21.15.138:0/3691833226 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d81f7380 con 0x55c3d81f3400 2024-04-27T05:23:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.716+0000 7f593378f640 1 --2- 172.21.15.138:0/3691833226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3400 0x55c3d84d4580 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x55c3d82fec00 tx=0x55c3d8304de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9ee5f8dfa09a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.716+0000 7f5932f8e640 1 -- 172.21.15.138:0/3691833226 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820bce0 con 0x55c3d81f3400 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.716+0000 7f5932f8e640 1 -- 172.21.15.138:0/3691833226 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d84cbb00 con 0x55c3d81f3400 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.716+0000 7f5932f8e640 1 -- 172.21.15.138:0/3691833226 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820be40 con 0x55c3d81f3400 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.716+0000 7f5937e30880 1 -- 172.21.15.138:0/3691833226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3400 msgr2=0x55c3d84d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.716+0000 7f5937e30880 1 --2- 172.21.15.138:0/3691833226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3400 0x55c3d84d4580 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x55c3d82fec00 tx=0x55c3d8304de0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 -- 172.21.15.138:0/3691833226 shutdown_connections 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 --2- 172.21.15.138:0/3691833226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3400 0x55c3d84d4580 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 -- 172.21.15.138:0/3691833226 >> 172.21.15.138:0/3691833226 conn(0x55c3d81f2c00 msgr2=0x55c3d82fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 -- 172.21.15.138:0/3691833226 shutdown_connections 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 -- 172.21.15.138:0/3691833226 wait complete. 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.717+0000 7f5937e30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d8232780 con 0x55c3d81f3000 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f5933f90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f5933f90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35134/0 (socket says 172.21.15.138:35134) 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f5933f90640 1 -- 172.21.15.138:0/380569211 learned_addr learned my addr 172.21.15.138:0/380569211 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f5933f90640 1 -- 172.21.15.138:0/380569211 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d81f6ea0 con 0x55c3d81f3000 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f5933f90640 1 --2- 172.21.15.138:0/380569211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55c3d90aecc0 tx=0x55c3d8304cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba041dabe08e7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f593278d640 1 -- 172.21.15.138:0/380569211 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820b4a0 con 0x55c3d81f3000 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f593278d640 1 -- 172.21.15.138:0/380569211 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d84cbe00 con 0x55c3d81f3000 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.718+0000 7f593278d640 1 -- 172.21.15.138:0/380569211 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820bce0 con 0x55c3d81f3000 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- 172.21.15.138:0/380569211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 msgr2=0x55c3d84d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 --2- 172.21.15.138:0/380569211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55c3d90aecc0 tx=0x55c3d8304cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- 172.21.15.138:0/380569211 shutdown_connections 2024-04-27T05:23:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 --2- 172.21.15.138:0/380569211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- 172.21.15.138:0/380569211 >> 172.21.15.138:0/380569211 conn(0x55c3d81f2c00 msgr2=0x55c3d82fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- 172.21.15.138:0/380569211 shutdown_connections 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- 172.21.15.138:0/380569211 wait complete. 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.719+0000 7f5937e30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d82328c0 con 0x55c3d81f3000 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5933f90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5933f90640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35150/0 (socket says 172.21.15.138:35150) 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5933f90640 1 -- 172.21.15.138:0/3161730291 learned_addr learned my addr 172.21.15.138:0/3161730291 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5933f90640 1 -- 172.21.15.138:0/3161730291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d81f76c0 con 0x55c3d81f3000 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5933f90640 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55c3d90aeea0 tx=0x55c3d8304c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5930f8a640 1 -- 172.21.15.138:0/3161730291 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820be40 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5930f8a640 1 -- 172.21.15.138:0/3161730291 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d90c4000 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c3d81f71e0 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5930f8a640 1 -- 172.21.15.138:0/3161730291 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820b4a0 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.720+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3d8264ea0 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.721+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.721+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.721+0000 7f5930f8a640 1 -- 172.21.15.138:0/3161730291 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c3d821d600 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.721+0000 7f5937e30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.721+0000 7f5937e30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d8216280 con 0x55c3d8feb800 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.721+0000 7f5930f8a640 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d9148000 0x55c3d84d5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5930f8a640 1 -- 172.21.15.138:0/3161730291 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55c3d8205a40 con 0x55c3d81f3000 2024-04-27T05:23:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5934791640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5933f90640 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d9148000 0x55c3d84d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5934791640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35154/0 (socket says 172.21.15.138:35154) 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5934791640 1 -- 172.21.15.138:0/1823370847 learned_addr learned my addr 172.21.15.138:0/1823370847 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5934791640 1 -- 172.21.15.138:0/1823370847 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d8264d00 con 0x55c3d8feb800 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5933f90640 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d9148000 0x55c3d84d5080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55c3d90f8450 tx=0x55c3d83054a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f5934791640 1 --2- 172.21.15.138:0/1823370847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55c3d90af470 tx=0x55c3d8305f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b27057291199d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f592e785640 1 -- 172.21.15.138:0/1823370847 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820be40 con 0x55c3d8feb800 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f592e785640 1 -- 172.21.15.138:0/1823370847 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d90c4300 con 0x55c3d8feb800 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.722+0000 7f592e785640 1 -- 172.21.15.138:0/1823370847 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820b4a0 con 0x55c3d8feb800 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- 172.21.15.138:0/1823370847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 msgr2=0x55c3d84d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 --2- 172.21.15.138:0/1823370847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55c3d90af470 tx=0x55c3d8305f80 comp rx=0 tx=0).stop 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- 172.21.15.138:0/1823370847 shutdown_connections 2024-04-27T05:23:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 --2- 172.21.15.138:0/1823370847 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- 172.21.15.138:0/1823370847 >> 172.21.15.138:0/1823370847 conn(0x55c3d81f3400 msgr2=0x55c3d82fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- 172.21.15.138:0/1823370847 shutdown_connections 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- 172.21.15.138:0/1823370847 wait complete. 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.723+0000 7f5937e30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d8217180 con 0x55c3d8feb800 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5934791640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5934791640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35162/0 (socket says 172.21.15.138:35162) 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 learned_addr learned my addr 172.21.15.138:0/721072117 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d8264b60 con 0x55c3d8feb800 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5934791640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55c3d8ff7470 tx=0x55c3d8305bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f592cf82640 1 -- 172.21.15.138:0/721072117 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820be40 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f592cf82640 1 -- 172.21.15.138:0/721072117 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d90c4480 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c3d82649c0 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f592cf82640 1 -- 172.21.15.138:0/721072117 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d820b4a0 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.724+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3d8264820 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f592cf82640 1 -- 172.21.15.138:0/721072117 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c3d821d8c0 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f592cf82640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8582000 0x55c3d84d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f592cf82640 1 -- 172.21.15.138:0/721072117 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55c3d8205c00 con 0x55c3d8feb800 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f593378f640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8582000 0x55c3d84d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c3d8febc00 0x55c3d84d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582400 con 0x55c3d8febc00 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.725+0000 7f5934791640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c3d8febc00 0x55c3d84d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f593378f640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8582000 0x55c3d84d5600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55c3d82fec00 tx=0x55c3d9170f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5934791640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c3d8febc00 0x55c3d84d5b80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 20 realm 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582800 con 0x55c3d8febc00 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.726+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582c00 con 0x55c3d8febc00 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583000 con 0x55c3d8febc00 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c3d8583400 0x55c3d84d6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582400 con 0x55c3d8583400 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5933f90640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c3d8583400 0x55c3d84d6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.727+0000 7f5933f90640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c3d8583400 0x55c3d84d6100 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5937e30880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5937e30880 20 searching for the correct realm 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582800 con 0x55c3d8583400 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5933f90640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c3d8582800 0x55c3d84d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582400 con 0x55c3d8582800 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f593378f640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c3d8582800 0x55c3d84d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.728+0000 7f593378f640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c3d8582800 0x55c3d84d6c00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.729+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.729+0000 7f593378f640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c3d8582400 0x55c3d84d7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.729+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.729+0000 7f5934791640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c3d8582400 0x55c3d84d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.729+0000 7f5934791640 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c3d8582400 0x55c3d84d7180 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.730+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582800 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.731+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.732+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.732+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.732+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.732+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582800 2024-04-27T05:23:19.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.732+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.732+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.733+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.734+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.735+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.735+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582800 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.735+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.735+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.735+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.735+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582800 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.736+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8583400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8febc00 2024-04-27T05:23:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fea800 con 0x55c3d8582800 2024-04-27T05:23:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5937e30880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:23:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5937e30880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:23:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5937e30880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5937e30880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.737+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582c00 con 0x55c3d8febc00 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583000 con 0x55c3d8febc00 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583800 con 0x55c3d8583400 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 zone default found 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 4 Realm: () 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 zonegroup default 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.738+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583c00 con 0x55c3d8582400 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6000 con 0x55c3d8583400 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6400 con 0x55c3d8583400 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 started sync module instance, tier type = 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:23:19.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.739+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6800 con 0x55c3d8582800 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.740+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.740+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94299635894528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6c00 con 0x55c3d8582800 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.740+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd7000 con 0x55c3d8febc00 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.741+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94299635894528] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.742+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.742+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94299635895680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd7400 con 0x55c3d8febc00 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.742+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd7800 con 0x55c3d8583400 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.743+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94299635895680] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.743+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.743+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94299635896832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6000 con 0x55c3d8583400 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.743+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6400 con 0x55c3d8582800 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.744+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.744+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94299635897984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd7800 con 0x55c3d8582800 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.744+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583800 con 0x55c3d8febc00 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.745+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94299635896832] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.746+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94299635897984] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.746+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.746+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94299635899136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8582c00 con 0x55c3d8febc00 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.746+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583000 con 0x55c3d8582400 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.747+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94299635899136] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.747+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.747+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94299635900288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8583c00 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.747+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d9148c00 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.748+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94299635900288] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.748+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.748+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94299635901440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6800 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.748+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6c00 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94299635901440] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94299635902592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd6400 con 0x55c3d8582400 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=0 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=1 2024-04-27T05:23:19.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=2 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=3 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=4 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=5 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.749+0000 7f5937e30880 20 add_watcher() i=6 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.750+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94299635902592] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.750+0000 7f5937e30880 20 add_watcher() i=7 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.750+0000 7f5937e30880 2 all 8 watchers are set, enabling cache 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.750+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd7800 con 0x55c3d8febc00 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.750+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.751+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94299635903744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d8fd7000 con 0x55c3d8febc00 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94299635903744] v31'234 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f591af5e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5937e30880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5937e30880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5937e30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.752+0000 7f5937e30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d8216780 con 0x55c3d8fd7c00 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f593378f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f593378f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35174/0 (socket says 172.21.15.138:35174) 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f593378f640 1 -- 172.21.15.138:0/1627809281 learned_addr learned my addr 172.21.15.138:0/1627809281 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f593378f640 1 -- 172.21.15.138:0/1627809281 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d8264680 con 0x55c3d8fd7c00 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f593378f640 1 --2- 172.21.15.138:0/1627809281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55c3d9172780 tx=0x55c3d9170fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de7c0813c825773 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f591a75d640 1 -- 172.21.15.138:0/1627809281 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d8fd8580 con 0x55c3d8fd7c00 2024-04-27T05:23:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f591a75d640 1 -- 172.21.15.138:0/1627809281 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d90c4000 con 0x55c3d8fd7c00 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.753+0000 7f591a75d640 1 -- 172.21.15.138:0/1627809281 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d8fd86e0 con 0x55c3d8fd7c00 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- 172.21.15.138:0/1627809281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 msgr2=0x55c3d84d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 --2- 172.21.15.138:0/1627809281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55c3d9172780 tx=0x55c3d9170fc0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- 172.21.15.138:0/1627809281 shutdown_connections 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 --2- 172.21.15.138:0/1627809281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- 172.21.15.138:0/1627809281 >> 172.21.15.138:0/1627809281 conn(0x55c3d8fd7400 msgr2=0x55c3d82fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- 172.21.15.138:0/1627809281 shutdown_connections 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- 172.21.15.138:0/1627809281 wait complete. 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 Processor -- start 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- start start 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.754+0000 7f5937e30880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d8216500 con 0x55c3d8fd7c00 2024-04-27T05:23:19.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f593378f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f593378f640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35180/0 (socket says 172.21.15.138:35180) 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f593378f640 1 -- 172.21.15.138:0/3668843629 learned_addr learned my addr 172.21.15.138:0/3668843629 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f593378f640 1 -- 172.21.15.138:0/3668843629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d82644e0 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f593378f640 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55c3d8565740 tx=0x55c3d9170e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f5918f5a640 1 -- 172.21.15.138:0/3668843629 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d8fd8840 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f5918f5a640 1 -- 172.21.15.138:0/3668843629 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3d90c4780 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c3d81f6ea0 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f5918f5a640 1 -- 172.21.15.138:0/3668843629 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c3d8fd8580 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.755+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3d81f76c0 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.756+0000 7f5918f5a640 1 -- 172.21.15.138:0/3668843629 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55c3d821cb00 con 0x55c3d8fd7c00 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.756+0000 7f5918f5a640 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8fd6800 0x55c3d84d7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.756+0000 7f5918f5a640 1 -- 172.21.15.138:0/3668843629 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55c3d8205dc0 con 0x55c3d8fd7c00 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.756+0000 7f5937e30880 5 note: GC not initialized 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.756+0000 7f5934791640 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8fd6800 0x55c3d84d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5916755640 20 reqs_thread_entry: start 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 20 init_complete bucket index max shards: 11 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5934791640 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8fd6800 0x55c3d84d7700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55c3d9172ba0 tx=0x55c3d8638000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5914751640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:50 2024 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5913f2f640 20 reqs_thread_entry: start 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866a000 con 0x55c3d8febc00 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.757+0000 7f5937e30880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.758+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866a400 con 0x55c3d8582400 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.759+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call] v31'35 uv35 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.759+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866a800 con 0x55c3d8582400 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.760+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(64 usage.10 [call] v31'36 uv35 ondisk = -61 ((61) No data available)) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.760+0000 7f591af5e640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.760+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94299635903744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866ac00 con 0x55c3d8febc00 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.762+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94299635903744] v31'235 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.762+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94299635894528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866b000 con 0x55c3d8582800 2024-04-27T05:23:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.763+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94299635894528] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.763+0000 7f5937e30880 20 remove_watcher() i=0 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.763+0000 7f5937e30880 2 removed watcher, disabling cache 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.763+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94299635895680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866b400 con 0x55c3d8febc00 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.764+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94299635895680] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.764+0000 7f5937e30880 20 remove_watcher() i=1 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.764+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94299635896832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866b800 con 0x55c3d8583400 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.765+0000 7f5933f90640 1 -- 172.21.15.138:0/721072117 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94299635896832] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598fc0 con 0x55c3d8583400 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.765+0000 7f5937e30880 20 remove_watcher() i=2 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.765+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94299635897984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866bc00 con 0x55c3d8582800 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.766+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94299635897984] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8599200 con 0x55c3d8582800 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.766+0000 7f5937e30880 20 remove_watcher() i=3 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.766+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94299635899136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866a000 con 0x55c3d8febc00 2024-04-27T05:23:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.767+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94299635899136] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8febc00 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.767+0000 7f5937e30880 20 remove_watcher() i=4 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.767+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94299635900288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866a400 con 0x55c3d8582400 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.769+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94299635900288] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.769+0000 7f5937e30880 20 remove_watcher() i=5 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.769+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94299635901440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866a800 con 0x55c3d8582400 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.770+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94299635901440] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.770+0000 7f5937e30880 20 remove_watcher() i=6 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.770+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94299635902592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c3d866ac00 con 0x55c3d8582400 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5934791640 1 -- 172.21.15.138:0/721072117 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94299635902592] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c3d8598900 con 0x55c3d8582400 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 20 remove_watcher() i=7 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c3d8582400 msgr2=0x55c3d84d7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55c3d8582400 0x55c3d84d7180 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c3d8febc00 msgr2=0x55c3d84d5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55c3d8febc00 0x55c3d84d5b80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c3d8582800 msgr2=0x55c3d84d6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55c3d8582800 0x55c3d84d6c00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c3d8583400 msgr2=0x55c3d84d6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55c3d8583400 0x55c3d84d6100 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8582000 msgr2=0x55c3d84d5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8582000 0x55c3d84d5600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55c3d82fec00 tx=0x55c3d9170f60 comp rx=0 tx=0).stop 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 msgr2=0x55c3d84d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/721072117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8feb800 0x55c3d84d4b00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55c3d8ff7470 tx=0x55c3d8305bc0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 reap_dead start 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f593378f640 1 -- 172.21.15.138:0/721072117 reap_dead start 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 shutdown_connections 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 >> 172.21.15.138:0/721072117 conn(0x55c3d81f3400 msgr2=0x55c3d82fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 shutdown_connections 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/721072117 wait complete. 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8fd6800 msgr2=0x55c3d84d7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.771+0000 7f5937e30880 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8fd6800 0x55c3d84d7700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55c3d9172ba0 tx=0x55c3d8638000 comp rx=0 tx=0).stop 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 msgr2=0x55c3d84d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55c3d8565740 tx=0x55c3d9170e40 comp rx=0 tx=0).stop 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 shutdown_connections 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d8fd6800 0x55c3d84d7700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3668843629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d8fd7c00 0x55c3d84d6680 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 >> 172.21.15.138:0/3668843629 conn(0x55c3d8fd7400 msgr2=0x55c3d82fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 shutdown_connections 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3668843629 wait complete. 2024-04-27T05:23:19.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d9148000 msgr2=0x55c3d84d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d9148000 0x55c3d84d5080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55c3d90f8450 tx=0x55c3d83054a0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 msgr2=0x55c3d84d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55c3d90aeea0 tx=0x55c3d8304c60 comp rx=0 tx=0).stop 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 shutdown_connections 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55c3d9148000 0x55c3d84d5080 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 --2- 172.21.15.138:0/3161730291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3d81f3000 0x55c3d84d4580 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 >> 172.21.15.138:0/3161730291 conn(0x55c3d81f2c00 msgr2=0x55c3d82fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 shutdown_connections 2024-04-27T05:23:19.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.772+0000 7f5937e30880 1 -- 172.21.15.138:0/3161730291 wait complete. 2024-04-27T05:23:19.791 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-27T05:23:19.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-27T05:23:19.791 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-27T05:23:19.835 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:19.835 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.833+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.833+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.833+0000 7ff083698880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.833+0000 7ff083698880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5625e0a24600 con 0x5625e074d400 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.833+0000 7ff083698880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5625e078c280 con 0x5625e074d000 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.834+0000 7ff081e3d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.834+0000 7ff08163c640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5625e074d400 0x5625e0854800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38246/0 (socket says 172.21.15.138:38246) 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.834+0000 7ff08163c640 1 -- 172.21.15.138:0/2830908749 learned_addr learned my addr 172.21.15.138:0/2830908749 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.834+0000 7ff081e3d640 1 -- 172.21.15.138:0/2830908749 >> v1:172.21.15.114:6789/0 conn(0x5625e074d400 legacy=0x5625e0854800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.834+0000 7ff081e3d640 1 -- 172.21.15.138:0/2830908749 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e0751380 con 0x5625e074d000 2024-04-27T05:23:19.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff081e3d640 1 --2- 172.21.15.138:0/2830908749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5625e0858d50 tx=0x5625e085ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b99ed545ef187f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff080e3b640 1 -- 172.21.15.138:0/2830908749 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0765ce0 con 0x5625e074d000 2024-04-27T05:23:19.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff080e3b640 1 -- 172.21.15.138:0/2830908749 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e0a25b00 con 0x5625e074d000 2024-04-27T05:23:19.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff080e3b640 1 -- 172.21.15.138:0/2830908749 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0765e40 con 0x5625e074d000 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff083698880 1 -- 172.21.15.138:0/2830908749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 msgr2=0x5625e0a2e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff083698880 1 --2- 172.21.15.138:0/2830908749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5625e0858d50 tx=0x5625e085ede0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff083698880 1 -- 172.21.15.138:0/2830908749 shutdown_connections 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff083698880 1 --2- 172.21.15.138:0/2830908749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.836+0000 7ff083698880 1 -- 172.21.15.138:0/2830908749 >> 172.21.15.138:0/2830908749 conn(0x5625e074cc00 msgr2=0x5625e0854000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.837+0000 7ff083698880 1 -- 172.21.15.138:0/2830908749 shutdown_connections 2024-04-27T05:23:19.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.837+0000 7ff083698880 1 -- 172.21.15.138:0/2830908749 wait complete. 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.837+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.837+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.837+0000 7ff083698880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.837+0000 7ff083698880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e078c780 con 0x5625e074d000 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff081e3d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff081e3d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35198/0 (socket says 172.21.15.138:35198) 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff081e3d640 1 -- 172.21.15.138:0/3896413527 learned_addr learned my addr 172.21.15.138:0/3896413527 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff081e3d640 1 -- 172.21.15.138:0/3896413527 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e0750ea0 con 0x5625e074d000 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff081e3d640 1 --2- 172.21.15.138:0/3896413527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5625e1551560 tx=0x5625e085ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e93b6d42d7a41d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff08063a640 1 -- 172.21.15.138:0/3896413527 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e07654a0 con 0x5625e074d000 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff08063a640 1 -- 172.21.15.138:0/3896413527 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e0a25e00 con 0x5625e074d000 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff08063a640 1 -- 172.21.15.138:0/3896413527 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0765ce0 con 0x5625e074d000 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff083698880 1 -- 172.21.15.138:0/3896413527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 msgr2=0x5625e0a2e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff083698880 1 --2- 172.21.15.138:0/3896413527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5625e1551560 tx=0x5625e085ecc0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff083698880 1 -- 172.21.15.138:0/3896413527 shutdown_connections 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff083698880 1 --2- 172.21.15.138:0/3896413527 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.838+0000 7ff083698880 1 -- 172.21.15.138:0/3896413527 >> 172.21.15.138:0/3896413527 conn(0x5625e074cc00 msgr2=0x5625e0855000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff083698880 1 -- 172.21.15.138:0/3896413527 shutdown_connections 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff083698880 1 -- 172.21.15.138:0/3896413527 wait complete. 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff083698880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff083698880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e078c8c0 con 0x5625e074d000 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff081e3d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff081e3d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35210/0 (socket says 172.21.15.138:35210) 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff081e3d640 1 -- 172.21.15.138:0/396604891 learned_addr learned my addr 172.21.15.138:0/396604891 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.839+0000 7ff081e3d640 1 -- 172.21.15.138:0/396604891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e07516c0 con 0x5625e074d000 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff081e3d640 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5625e1608de0 tx=0x5625e085ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff07ee37640 1 -- 172.21.15.138:0/396604891 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0765e40 con 0x5625e074d000 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff07ee37640 1 -- 172.21.15.138:0/396604891 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e152c000 con 0x5625e074d000 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff07ee37640 1 -- 172.21.15.138:0/396604891 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e07654a0 con 0x5625e074d000 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625e07511e0 con 0x5625e074d000 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625e07beea0 con 0x5625e074d000 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff083698880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff083698880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e078cc80 con 0x5625e1544c00 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff07ee37640 1 -- 172.21.15.138:0/396604891 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5625e0776dc0 con 0x5625e074d000 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.840+0000 7ff08263e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff08263e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35216/0 (socket says 172.21.15.138:35216) 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff08263e640 1 -- 172.21.15.138:0/4159612243 learned_addr learned my addr 172.21.15.138:0/4159612243 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff07ee37640 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e1545c00 0x5625e0a2f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff07ee37640 1 -- 172.21.15.138:0/396604891 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5625e075fa40 con 0x5625e074d000 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff081e3d640 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e1545c00 0x5625e0a2f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff08263e640 1 -- 172.21.15.138:0/4159612243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e07bed00 con 0x5625e1544c00 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff08263e640 1 --2- 172.21.15.138:0/4159612243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5625e1651080 tx=0x5625e085fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c880de9be74ddb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff081e3d640 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e1545c00 0x5625e0a2f080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x5625e1650300 tx=0x5625e085fd40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff07c632640 1 -- 172.21.15.138:0/4159612243 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0765e40 con 0x5625e1544c00 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff07c632640 1 -- 172.21.15.138:0/4159612243 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e152c600 con 0x5625e1544c00 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff07c632640 1 -- 172.21.15.138:0/4159612243 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e07654a0 con 0x5625e1544c00 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff083698880 1 -- 172.21.15.138:0/4159612243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 msgr2=0x5625e0a2eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff083698880 1 --2- 172.21.15.138:0/4159612243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5625e1651080 tx=0x5625e085fbc0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff083698880 1 -- 172.21.15.138:0/4159612243 shutdown_connections 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff083698880 1 --2- 172.21.15.138:0/4159612243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.841+0000 7ff083698880 1 -- 172.21.15.138:0/4159612243 >> 172.21.15.138:0/4159612243 conn(0x5625e074d400 msgr2=0x5625e0857000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff083698880 1 -- 172.21.15.138:0/4159612243 shutdown_connections 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff083698880 1 -- 172.21.15.138:0/4159612243 wait complete. 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff083698880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff083698880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e0770280 con 0x5625e1544c00 2024-04-27T05:23:19.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff08263e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff08263e640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35230/0 (socket says 172.21.15.138:35230) 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 learned_addr learned my addr 172.21.15.138:0/3842240357 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.842+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e07beb60 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.843+0000 7ff08263e640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5625e1651f50 tx=0x5625e085f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.843+0000 7ff07ae2f640 1 -- 172.21.15.138:0/3842240357 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0765e40 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.843+0000 7ff07ae2f640 1 -- 172.21.15.138:0/3842240357 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e152c780 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.843+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625e07be9c0 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.843+0000 7ff07ae2f640 1 -- 172.21.15.138:0/3842240357 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e07654a0 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.843+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625e07be820 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff07ae2f640 1 -- 172.21.15.138:0/3842240357 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5625e0776b00 con 0x5625e1544c00 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff07ae2f640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e0ace000 0x5625e0a2f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff07ae2f640 1 -- 172.21.15.138:0/3842240357 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5625e075fc00 con 0x5625e1544c00 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff08163c640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e0ace000 0x5625e0a2f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5625e1545000 0x5625e0a2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e1545400 con 0x5625e1545000 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff08263e640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5625e1545000 0x5625e0a2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff08163c640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e0ace000 0x5625e0a2f600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5625e1694ea0 tx=0x5625e16bcd80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.844+0000 7ff08263e640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5625e1545000 0x5625e0a2fb80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 realm 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e1545800 con 0x5625e1545000 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acf000 con 0x5625e1545000 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acf400 con 0x5625e1545000 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5625e0acf800 0x5625e0a30100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acfc00 con 0x5625e0acf800 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.845+0000 7ff081e3d640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5625e0acf800 0x5625e0a30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff081e3d640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5625e0acf800 0x5625e0a30100 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff083698880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff083698880 20 searching for the correct realm 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acf000 con 0x5625e0acf800 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff081e3d640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5625e0acf000 0x5625e0a30c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acfc00 con 0x5625e0acf000 2024-04-27T05:23:19.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.846+0000 7ff08163c640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5625e0acf000 0x5625e0a30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.847+0000 7ff08163c640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5625e0acf000 0x5625e0a30c00 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.847+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.847+0000 7ff08163c640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5625e0acfc00 0x5625e0a31180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.847+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.847+0000 7ff08263e640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5625e0acfc00 0x5625e0a31180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.847+0000 7ff08263e640 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5625e0acfc00 0x5625e0a31180 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.848+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf000 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.849+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf000 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.850+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.851+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf000 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.852+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf000 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf800 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.853+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acfc00 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e1545000 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acec00 con 0x5625e0acf000 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acf400 con 0x5625e1545000 2024-04-27T05:23:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.854+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e1545400 con 0x5625e1545000 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e1545800 con 0x5625e0acf800 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 zone default found 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 4 Realm: () 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:23:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 zonegroup default 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee000 con 0x5625e0acfc00 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee400 con 0x5625e0acf800 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.855+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee800 con 0x5625e0acf800 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.856+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.856+0000 7ff083698880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.856+0000 7ff083698880 20 started sync module instance, tier type = 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.856+0000 7ff083698880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.856+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aeec00 con 0x5625e0acf000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.857+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.857+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94720682518784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aef000 con 0x5625e0acf000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.857+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aef400 con 0x5625e1545000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.858+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94720682518784] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.858+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.858+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94720682519936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aef800 con 0x5625e1545000 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.858+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aefc00 con 0x5625e0acf800 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.859+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.859+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94720682521088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee400 con 0x5625e0acf800 2024-04-27T05:23:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.859+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee800 con 0x5625e0acf000 2024-04-27T05:23:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.859+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94720682519936] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.860+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94720682521088] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.860+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.860+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94720682522240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aefc00 con 0x5625e0acf000 2024-04-27T05:23:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.860+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee000 con 0x5625e1545000 2024-04-27T05:23:19.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.861+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.861+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94720682523392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e1545800 con 0x5625e1545000 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.861+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e1545400 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.861+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94720682522240] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.862+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.862+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94720682523392] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.862+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94720682524544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0acf400 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.862+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e16ab800 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.863+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94720682524544] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.864+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.864+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94720682525696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e16ab400 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.864+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aeec00 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94720682525696] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94720682526848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aef000 con 0x5625e0acfc00 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=0 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=1 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=2 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=3 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=4 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=5 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.865+0000 7ff083698880 20 add_watcher() i=6 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.866+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94720682526848] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.866+0000 7ff083698880 20 add_watcher() i=7 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.866+0000 7ff083698880 2 all 8 watchers are set, enabling cache 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.866+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aee800 con 0x5625e1545000 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.866+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.866+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94720682528000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0aef400 con 0x5625e1545000 2024-04-27T05:23:19.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.867+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94720682528000] v31'236 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.867+0000 7ff068e0b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.867+0000 7ff083698880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.867+0000 7ff083698880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff083698880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff083698880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e07708c0 con 0x5625e0acec00 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff08163c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff08163c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35236/0 (socket says 172.21.15.138:35236) 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff08163c640 1 -- 172.21.15.138:0/1877190861 learned_addr learned my addr 172.21.15.138:0/1877190861 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.868+0000 7ff08163c640 1 -- 172.21.15.138:0/1877190861 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e07be680 con 0x5625e0acec00 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff08163c640 1 --2- 172.21.15.138:0/1877190861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x5625e0aac930 tx=0x5625e16bcd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c381179c926941c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff06860a640 1 -- 172.21.15.138:0/1877190861 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0af0580 con 0x5625e0acec00 2024-04-27T05:23:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff06860a640 1 -- 172.21.15.138:0/1877190861 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e152c000 con 0x5625e0acec00 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff06860a640 1 -- 172.21.15.138:0/1877190861 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0af06e0 con 0x5625e0acec00 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff083698880 1 -- 172.21.15.138:0/1877190861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 msgr2=0x5625e0a30680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff083698880 1 --2- 172.21.15.138:0/1877190861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x5625e0aac930 tx=0x5625e16bcd20 comp rx=0 tx=0).stop 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff083698880 1 -- 172.21.15.138:0/1877190861 shutdown_connections 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff083698880 1 --2- 172.21.15.138:0/1877190861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff083698880 1 -- 172.21.15.138:0/1877190861 >> 172.21.15.138:0/1877190861 conn(0x5625e0aef800 msgr2=0x5625e0857800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.869+0000 7ff083698880 1 -- 172.21.15.138:0/1877190861 shutdown_connections 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff083698880 1 -- 172.21.15.138:0/1877190861 wait complete. 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff083698880 1 Processor -- start 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff083698880 1 -- start start 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff083698880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff083698880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625e0770780 con 0x5625e0acec00 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff08163c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff08163c640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35238/0 (socket says 172.21.15.138:35238) 2024-04-27T05:23:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff08163c640 1 -- 172.21.15.138:0/125068547 learned_addr learned my addr 172.21.15.138:0/125068547 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.870+0000 7ff08163c640 1 -- 172.21.15.138:0/125068547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625e07be4e0 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff08163c640 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x5625e0aad770 tx=0x5625e16bd1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff066e07640 1 -- 172.21.15.138:0/125068547 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0af0840 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff066e07640 1 -- 172.21.15.138:0/125068547 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625e152ca80 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625e0751380 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff066e07640 1 -- 172.21.15.138:0/125068547 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625e0af0580 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625e0750ea0 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.871+0000 7ff066e07640 1 -- 172.21.15.138:0/125068547 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5625e07778c0 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff066e07640 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e16ab800 0x5625e0a31700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff066e07640 1 -- 172.21.15.138:0/125068547 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5625e075fdc0 con 0x5625e0acec00 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff083698880 5 note: GC not initialized 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff08263e640 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e16ab800 0x5625e0a31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff064602640 20 reqs_thread_entry: start 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff083698880 20 init_complete bucket index max shards: 11 2024-04-27T05:23:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff08263e640 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e16ab800 0x5625e0a31700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5625e0aad170 tx=0x5625e16bd020 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff083698880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff0625fe640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:50 2024 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff061ddc640 20 reqs_thread_entry: start 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff083698880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.872+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4000 con 0x5625e1545000 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.873+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.873+0000 7ff083698880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.873+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4400 con 0x5625e0acfc00 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.873+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 152+0+15 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.874+0000 7ff068e0b640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.874+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94720682528000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4800 con 0x5625e1545000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.875+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94720682528000] v31'237 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.875+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94720682518784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4c00 con 0x5625e0acf000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.876+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94720682518784] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.876+0000 7ff083698880 20 remove_watcher() i=0 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.876+0000 7ff083698880 2 removed watcher, disabling cache 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.876+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94720682519936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc5000 con 0x5625e1545000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.877+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94720682519936] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.877+0000 7ff083698880 20 remove_watcher() i=1 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.877+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94720682521088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc5400 con 0x5625e0acf800 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.878+0000 7ff081e3d640 1 -- 172.21.15.138:0/3842240357 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94720682521088] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8fc0 con 0x5625e0acf800 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.878+0000 7ff083698880 20 remove_watcher() i=2 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.878+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94720682522240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc5800 con 0x5625e0acf000 2024-04-27T05:23:19.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.879+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94720682522240] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad9200 con 0x5625e0acf000 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.879+0000 7ff083698880 20 remove_watcher() i=3 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.879+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94720682523392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc5c00 con 0x5625e1545000 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.880+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94720682523392] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e1545000 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.880+0000 7ff083698880 20 remove_watcher() i=4 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.880+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94720682524544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4000 con 0x5625e0acfc00 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.881+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94720682524544] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.881+0000 7ff083698880 20 remove_watcher() i=5 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.881+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94720682525696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4400 con 0x5625e0acfc00 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.882+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94720682525696] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.882+0000 7ff083698880 20 remove_watcher() i=6 2024-04-27T05:23:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.882+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94720682526848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5625e0bc4800 con 0x5625e0acfc00 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff08263e640 1 -- 172.21.15.138:0/3842240357 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94720682526848] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5625e0ad8900 con 0x5625e0acfc00 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 20 remove_watcher() i=7 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5625e0acfc00 msgr2=0x5625e0a31180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x5625e0acfc00 0x5625e0a31180 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5625e1545000 msgr2=0x5625e0a2fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5625e1545000 0x5625e0a2fb80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5625e0acf000 msgr2=0x5625e0a30c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x5625e0acf000 0x5625e0a30c00 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5625e0acf800 msgr2=0x5625e0a30100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.883+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5625e0acf800 0x5625e0a30100 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e0ace000 msgr2=0x5625e0a2f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e0ace000 0x5625e0a2f600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5625e1694ea0 tx=0x5625e16bcd80 comp rx=0 tx=0).stop 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 msgr2=0x5625e0a2eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 --2- 172.21.15.138:0/3842240357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e1544c00 0x5625e0a2eb00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5625e1651f50 tx=0x5625e085f9e0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 reap_dead start 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 shutdown_connections 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 >> 172.21.15.138:0/3842240357 conn(0x5625e074d400 msgr2=0x5625e0857000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff08163c640 1 -- 172.21.15.138:0/3842240357 reap_dead start 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 shutdown_connections 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/3842240357 wait complete. 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e16ab800 msgr2=0x5625e0a31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e16ab800 0x5625e0a31700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5625e0aad170 tx=0x5625e16bd020 comp rx=0 tx=0).stop 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 msgr2=0x5625e0a30680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x5625e0aad770 tx=0x5625e16bd1a0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 shutdown_connections 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e16ab800 0x5625e0a31700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 --2- 172.21.15.138:0/125068547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e0acec00 0x5625e0a30680 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 >> 172.21.15.138:0/125068547 conn(0x5625e0aef800 msgr2=0x5625e0857800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 shutdown_connections 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/125068547 wait complete. 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.884+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e1545c00 msgr2=0x5625e0a2f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e1545c00 0x5625e0a2f080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x5625e1650300 tx=0x5625e085fd40 comp rx=0 tx=0).stop 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 msgr2=0x5625e0a2e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5625e1608de0 tx=0x5625e085ec60 comp rx=0 tx=0).stop 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 shutdown_connections 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5625e1545c00 0x5625e0a2f080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 --2- 172.21.15.138:0/396604891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5625e074d000 0x5625e0a2e580 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 >> 172.21.15.138:0/396604891 conn(0x5625e074cc00 msgr2=0x5625e0855000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 shutdown_connections 2024-04-27T05:23:19.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.885+0000 7ff083698880 1 -- 172.21.15.138:0/396604891 wait complete. 2024-04-27T05:23:19.907 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:19.907 INFO:teuthology.orchestra.run.smithi138.stdout: "entries": [], 2024-04-27T05:23:19.907 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": [] 2024-04-27T05:23:19.907 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:19.908 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-27T05:23:19.908 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-27T05:23:19.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-27T05:23:19.908 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-04-27T05:23:19.951 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:19.951 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.949+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.949+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7d1b26880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7d1b26880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bc0bd9c600 con 0x55bc0bac5400 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7d1b26880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bc0bb04280 con 0x55bc0bac5000 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7ce3d4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7ce3d4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35246/0 (socket says 172.21.15.138:35246) 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7cdbd3640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55bc0bac5400 0x55bc0bbcc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38258/0 (socket says 172.21.15.138:38258) 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/3427671644 learned_addr learned my addr 172.21.15.138:0/3427671644 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/3427671644 >> v1:172.21.15.114:6789/0 conn(0x55bc0bac5400 legacy=0x55bc0bbcc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.950+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/3427671644 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bac9380 con 0x55bc0bac5000 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/3427671644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55bc0bbd0d50 tx=0x55bc0bbd6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5710050a60907c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7cd3d2640 1 -- 172.21.15.138:0/3427671644 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0baddce0 con 0x55bc0bac5000 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7cd3d2640 1 -- 172.21.15.138:0/3427671644 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0bd9db00 con 0x55bc0bac5000 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7cd3d2640 1 -- 172.21.15.138:0/3427671644 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badde40 con 0x55bc0bac5000 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3427671644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 msgr2=0x55bc0bda6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/3427671644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55bc0bbd0d50 tx=0x55bc0bbd6de0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3427671644 shutdown_connections 2024-04-27T05:23:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/3427671644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3427671644 >> 172.21.15.138:0/3427671644 conn(0x55bc0bac4c00 msgr2=0x55bc0bbcc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3427671644 shutdown_connections 2024-04-27T05:23:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.952+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3427671644 wait complete. 2024-04-27T05:23:19.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.953+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.953+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.953+0000 7fd7d1b26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.953+0000 7fd7d1b26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0bb04780 con 0x55bc0bac5000 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ce3d4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ce3d4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35262/0 (socket says 172.21.15.138:35262) 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2931153519 learned_addr learned my addr 172.21.15.138:0/2931153519 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2931153519 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bac8ea0 con 0x55bc0bac5000 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2931153519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x55bc0c8c9560 tx=0x55bc0bbd6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c36ed3ac3482fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ccbd1640 1 -- 172.21.15.138:0/2931153519 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badd4a0 con 0x55bc0bac5000 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ccbd1640 1 -- 172.21.15.138:0/2931153519 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0bd9de00 con 0x55bc0bac5000 2024-04-27T05:23:19.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7ccbd1640 1 -- 172.21.15.138:0/2931153519 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0baddce0 con 0x55bc0bac5000 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2931153519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 msgr2=0x55bc0bda6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2931153519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x55bc0c8c9560 tx=0x55bc0bbd6cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2931153519 shutdown_connections 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2931153519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2931153519 >> 172.21.15.138:0/2931153519 conn(0x55bc0bac4c00 msgr2=0x55bc0bbcd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.954+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2931153519 shutdown_connections 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2931153519 wait complete. 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7d1b26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7d1b26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0bb048c0 con 0x55bc0bac5000 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7ce3d4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7ce3d4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35272/0 (socket says 172.21.15.138:35272) 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2372280444 learned_addr learned my addr 172.21.15.138:0/2372280444 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2372280444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bac96c0 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.955+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x55bc0c9812c0 tx=0x55bc0bbd6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7cb3ce640 1 -- 172.21.15.138:0/2372280444 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badde40 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7cb3ce640 1 -- 172.21.15.138:0/2372280444 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0c8a4000 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc0bac91e0 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7cb3ce640 1 -- 172.21.15.138:0/2372280444 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badd4a0 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0bb36ea0 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7cb3ce640 1 -- 172.21.15.138:0/2372280444 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bc0baeeb00 con 0x55bc0bac5000 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7d1b26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.956+0000 7fd7d1b26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0bae8280 con 0x55bc0c8bcc00 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cb3ce640 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0c8bdc00 0x55bc0bda7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cb3ce640 1 -- 172.21.15.138:0/2372280444 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bc0bad7a40 con 0x55bc0bac5000 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cebd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cebd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35284/0 (socket says 172.21.15.138:35284) 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1389683858 learned_addr learned my addr 172.21.15.138:0/1389683858 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0c8bdc00 0x55bc0bda7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1389683858 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bb36d00 con 0x55bc0c8bcc00 2024-04-27T05:23:19.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0c8bdc00 0x55bc0bda7080 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55bc0c9c6180 tx=0x55bc0bbd7c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/1389683858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55bc0c981e60 tx=0x55bc0bbd7f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5066131f7a8d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7c8bc9640 1 -- 172.21.15.138:0/1389683858 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badde40 con 0x55bc0c8bcc00 2024-04-27T05:23:19.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7c8bc9640 1 -- 172.21.15.138:0/1389683858 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0c8a4600 con 0x55bc0c8bcc00 2024-04-27T05:23:19.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.957+0000 7fd7c8bc9640 1 -- 172.21.15.138:0/1389683858 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badd4a0 con 0x55bc0c8bcc00 2024-04-27T05:23:19.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1389683858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 msgr2=0x55bc0bda6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:19.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1389683858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55bc0c981e60 tx=0x55bc0bbd7f80 comp rx=0 tx=0).stop 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1389683858 shutdown_connections 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1389683858 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1389683858 >> 172.21.15.138:0/1389683858 conn(0x55bc0bac5400 msgr2=0x55bc0bbcf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1389683858 shutdown_connections 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1389683858 wait complete. 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:19.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7d1b26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0bae9180 con 0x55bc0c8bcc00 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7cebd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7cebd5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35298/0 (socket says 172.21.15.138:35298) 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.958+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 learned_addr learned my addr 172.21.15.138:0/1492183943 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bb36b60 con 0x55bc0c8bcc00 2024-04-27T05:23:19.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55bc0c980b40 tx=0x55bc0bbd7b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7c73c6640 1 -- 172.21.15.138:0/1492183943 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badde40 con 0x55bc0c8bcc00 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7c73c6640 1 -- 172.21.15.138:0/1492183943 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0c8a4780 con 0x55bc0c8bcc00 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc0bb369c0 con 0x55bc0c8bcc00 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7c73c6640 1 -- 172.21.15.138:0/1492183943 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0badd4a0 con 0x55bc0c8bcc00 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.959+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0bb36820 con 0x55bc0c8bcc00 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7c73c6640 1 -- 172.21.15.138:0/1492183943 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bc0baef600 con 0x55bc0c8bcc00 2024-04-27T05:23:19.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7c73c6640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0be46000 0x55bc0bda7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7c73c6640 1 -- 172.21.15.138:0/1492183943 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bc0bad7c00 con 0x55bc0c8bcc00 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0c8bd000 0x55bc0bda7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0be46000 0x55bc0bda7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0c8bd000 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0c8bd000 0x55bc0bda7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0be46000 0x55bc0bda7600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55bc0bbd1380 tx=0x55bc0ca22f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0c8bd000 0x55bc0bda7b80 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.960+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 realm 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd800 con 0x55bc0c8bd000 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:19.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47000 con 0x55bc0c8bd000 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47400 con 0x55bc0c8bd000 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0be47800 0x55bc0bda8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47c00 con 0x55bc0be47800 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.961+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0be47800 0x55bc0bda8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0be47800 0x55bc0bda8100 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:19.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7d1b26880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7d1b26880 20 searching for the correct realm 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47000 con 0x55bc0be47800 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0be47000 0x55bc0bda8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47c00 con 0x55bc0be47000 2024-04-27T05:23:19.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0be47000 0x55bc0bda8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.962+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0be47000 0x55bc0bda8c00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0be47c00 0x55bc0bda9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0be47c00 0x55bc0bda9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0be47c00 0x55bc0bda9180 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.963+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.964+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47000 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.965+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47000 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.966+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.967+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47000 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.968+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47000 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47800 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47c00 2024-04-27T05:23:20.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.969+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be46c00 con 0x55bc0be47000 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:23:20.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47400 con 0x55bc0c8bd000 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0c8bd000 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:20.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd800 con 0x55bc0be47800 2024-04-27T05:23:20.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:23:20.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 zone default found 2024-04-27T05:23:20.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 4 Realm: () 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 zonegroup default 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.970+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68000 con 0x55bc0be47c00 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68400 con 0x55bc0be47800 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68800 con 0x55bc0be47800 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 started sync module instance, tier type = 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.971+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68c00 con 0x55bc0be47000 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.972+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.972+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94266141042944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69000 con 0x55bc0be47000 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.972+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69400 con 0x55bc0c8bd000 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.973+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94266141042944] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.974+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.974+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94266141044096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69800 con 0x55bc0c8bd000 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.974+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69c00 con 0x55bc0be47800 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.975+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94266141044096] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.975+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.975+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94266141045248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68400 con 0x55bc0be47800 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.975+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68800 con 0x55bc0be47000 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.976+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.976+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94266141046400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69c00 con 0x55bc0be47000 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.976+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68000 con 0x55bc0c8bd000 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.976+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94266141045248] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.977+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94266141046400] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.978+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.978+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94266141047552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd800 con 0x55bc0c8bd000 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.978+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.979+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94266141047552] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.979+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.979+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94266141048704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be47400 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.979+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0ca24c00 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.980+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94266141048704] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.982+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.982+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94266141049856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0ca24800 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.982+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68c00 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94266141049856] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94266141051008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69000 con 0x55bc0be47c00 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=0 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=1 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=2 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=3 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=4 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=5 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.983+0000 7fd7d1b26880 20 add_watcher() i=6 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.984+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94266141051008] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.984+0000 7fd7d1b26880 20 add_watcher() i=7 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.984+0000 7fd7d1b26880 2 all 8 watchers are set, enabling cache 2024-04-27T05:23:20.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.984+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68800 con 0x55bc0c8bd000 2024-04-27T05:23:20.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.984+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.984+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94266141052160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69400 con 0x55bc0c8bd000 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.985+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94266141052160] v31'238 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.985+0000 7fd7b53a2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.985+0000 7fd7d1b26880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.985+0000 7fd7d1b26880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.985+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.985+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7d1b26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7d1b26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0bae8780 con 0x55bc0be46c00 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7cdbd3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7cdbd3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35308/0 (socket says 172.21.15.138:35308) 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/3727028405 learned_addr learned my addr 172.21.15.138:0/3727028405 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/3727028405 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bb36680 con 0x55bc0be46c00 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.986+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/3727028405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55bc0be24ab0 tx=0x55bc0ca22f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4c9a708d2f024e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7b4ba1640 1 -- 172.21.15.138:0/3727028405 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0be6a580 con 0x55bc0be46c00 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7b4ba1640 1 -- 172.21.15.138:0/3727028405 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0c8a4000 con 0x55bc0be46c00 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7b4ba1640 1 -- 172.21.15.138:0/3727028405 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0be6a6e0 con 0x55bc0be46c00 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3727028405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 msgr2=0x55bc0bda8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/3727028405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x55bc0be24ab0 tx=0x55bc0ca22f00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3727028405 shutdown_connections 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/3727028405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3727028405 >> 172.21.15.138:0/3727028405 conn(0x55bc0be69800 msgr2=0x55bc0bbcf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3727028405 shutdown_connections 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.987+0000 7fd7d1b26880 1 -- 172.21.15.138:0/3727028405 wait complete. 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7d1b26880 1 Processor -- start 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7d1b26880 1 -- start start 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7d1b26880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7d1b26880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0bae8500 con 0x55bc0be46c00 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7cdbd3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7cdbd3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35316/0 (socket says 172.21.15.138:35316) 2024-04-27T05:23:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 learned_addr learned my addr 172.21.15.138:0/2637311291 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0bb364e0 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.988+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55bc0be25800 tx=0x55bc0ca23080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7b339e640 1 -- 172.21.15.138:0/2637311291 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0be6a840 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7b339e640 1 -- 172.21.15.138:0/2637311291 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc0c8a4a80 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7b339e640 1 -- 172.21.15.138:0/2637311291 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc0be6a580 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc0bac9380 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0bac8ea0 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7b339e640 1 -- 172.21.15.138:0/2637311291 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bc0baefb80 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7b339e640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0ca24c00 0x55bc0bda9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.989+0000 7fd7b339e640 1 -- 172.21.15.138:0/2637311291 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bc0bad7dc0 con 0x55bc0be46c00 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.990+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0ca24c00 0x55bc0bda9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.990+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0ca24800 0x55bc0bf06000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.990+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a000 con 0x55bc0ca24800 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.990+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0ca24800 0x55bc0bf06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.990+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0ca24c00 0x55bc0bda9700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55bc0c9c6f90 tx=0x55bc0ca23e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.990+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0ca24800 0x55bc0bf06000 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.991+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v31'62 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0ca24800 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.991+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0bf0a400 0x55bc0bf06580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.991+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a800 con 0x55bc0bf0a400 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.991+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0bf0a400 0x55bc0bf06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.992+0000 7fd7cdbd3640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0bf0a400 0x55bc0bf06580 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.993+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v31'37 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.993+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0bf0ac00 0x55bc0bf06b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.993+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b000 con 0x55bc0bf0ac00 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.993+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0bf0ac00 0x55bc0bf06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.993+0000 7fd7cebd5640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0bf0ac00 0x55bc0bf06b00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.994+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.994+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b400 con 0x55bc0ca24800 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.995+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(4 gc.3 [create,call,call,call] v31'239 uv168 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0ca24800 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.995+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0bf0b800 0x55bc0bf07080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.995+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0bc00 con 0x55bc0bf0b800 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.995+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0bf0b800 0x55bc0bf07080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.995+0000 7fd7ce3d4640 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0bf0b800 0x55bc0bf07080 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.996+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(5 gc.4 [create,call,call,call] v31'80 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.996+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22000 con 0x55bc0bf0ac00 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.997+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v31'64 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.997+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22400 con 0x55bc0ca24800 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.998+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v31'37 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0ca24800 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.998+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22800 con 0x55bc0bf0a400 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.999+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v31'55 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:19.999+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22c00 con 0x55bc0bf0b800 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.000+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(9 gc.8 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.000+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23000 con 0x55bc0bf0ac00 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.001+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(10 gc.9 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.001+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23400 con 0x55bc0bf0b800 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.002+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(11 gc.10 [create,call,call,call] v31'81 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.002+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23800 con 0x55bc0bf0a400 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.003+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v31'37 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.003+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23c00 con 0x55bc0bf0b800 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.004+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(13 gc.12 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.004+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22400 con 0x55bc0bf0a400 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.005+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(14 gc.13 [create,call,call,call] v31'75 uv57 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.005+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22c00 con 0x55bc0ca24800 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.006+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v31'240 uv170 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0ca24800 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.006+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.007+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(16 gc.15 [create,call,call,call] v31'80 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.007+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a000 con 0x55bc0bf0b800 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.008+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(17 gc.16 [create,call,call,call] v31'82 uv61 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.008+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.009+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(18 gc.17 [create,call,call,call] v31'40 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.009+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0bc00 con 0x55bc0ca24800 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.010+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(19 gc.18 [create,call,call,call] v31'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0ca24800 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.010+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68400 con 0x55bc0ca24800 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.011+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(20 gc.19 [create,call,call,call] v31'241 uv172 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0ca24800 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.011+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22000 con 0x55bc0bf0ac00 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.012+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 gc.20 [create,call,call,call] v31'81 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.012+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23000 con 0x55bc0bf0b800 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.013+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(22 gc.21 [create,call,call,call] v31'44 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.013+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.014+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 gc.22 [create,call,call,call] v31'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.014+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22800 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.015+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(24 gc.23 [create,call,call,call] v31'46 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.015+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf23800 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.016+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(25 gc.24 [create,call,call,call] v31'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.016+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf22400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.016+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(26 gc.25 [create,call,call,call] v31'65 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.016+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b000 con 0x55bc0bf0a400 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.018+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(27 gc.26 [create,call,call,call] v31'37 uv30 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.018+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.019+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(28 gc.27 [create,call,call,call] v31'41 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.019+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a800 con 0x55bc0bf0a400 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.020+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v31'56 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.020+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.021+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(30 gc.29 [create,call,call,call] v31'82 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.021+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68c00 con 0x55bc0bf0a400 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.022+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/2637311291 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v31'76 uv59 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0bf0a400 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.022+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69000 con 0x55bc0bf0b800 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(32 gc.31 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0bf0b800 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7b0b99640 20 reqs_thread_entry: start 2024-04-27T05:23:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7d1b26880 20 init_complete bucket index max shards: 11 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7d1b26880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7aeb95640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:50 2024 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7ae373640 20 reqs_thread_entry: start 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7d1b26880 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.023+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92000 con 0x55bc0c8bd000 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 20 rados_obj.operate() r=0 bl.length=357 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T05:23:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92400 con 0x55bc0be47000 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv41 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 : miss 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.024+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:64 8.11 8:8d002013:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.6:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92800 con 0x55bc0be47800 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(64 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 20 rados_obj.operate() r=0 bl.length=352 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 info.flags=0x17 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 10 adding default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 to cache LRU end 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:23:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 4.3c 4:3ff255e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(65 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:66 4.a 4:507ee1fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93000 con 0x55bc0be47000 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(66 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.025+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:67 4.2d 4:b4b4270b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93400 con 0x55bc0be47c00 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.026+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(67 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.026+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:68 4.7 4:e3c21a1d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93800 con 0x55bc0be47c00 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.026+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(68 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.026+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 4.29 4:967bb226:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93c00 con 0x55bc0be47800 2024-04-27T05:23:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(69 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 [call out=425b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7d1b26880 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:70 4.10 4:08b73cb8:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92800 con 0x55bc0be47000 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 13 ==== osd_op_reply(70 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 4.a 4:53835284:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92000 con 0x55bc0be47000 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 14 ==== osd_op_reply(71 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 4.2f 4:f74f9645:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0c8bd000 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(72 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.027+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 4.15 4:a95d8a74:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a000 con 0x55bc0be47800 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== osd_op_reply(73 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:74 4.5 4:a38b0747:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0bc00 con 0x55bc0be47800 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(74 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 4.f 4:f1d029e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b400 con 0x55bc0c8bd000 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(75 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7d1b26880 20 get_obj_state: rctx=0x55bc0bf72230 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55bc0bf961e8 s->prefetch_data=0 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.028+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:76 3.16 3:689a7301:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.6_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68400 con 0x55bc0be47000 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 15 ==== osd_op_reply(76 2f19b8ff-2608-4d86-8f1c-430623205476.4231.6_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=877b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7d1b26880 10 manifest: total_size = 6 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7d1b26880 20 get_obj_state: setting s->obj_tag to 2f19b8ff-2608-4d86-8f1c-430623205476.4229.9394310414318619580 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7d1b26880 20 get_obj_state: rctx=0x55bc0bf72230 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55bc0bf961e8 s->prefetch_data=0 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7d1b26880 20 prepare_atomic_modification: state is not atomic. state=0x55bc0bf961e8 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7d1b26880 20 bucket index object: default.rgw.buckets.index:.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.029+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:77 4.29 4:967bb226:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69000 con 0x55bc0be47800 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.030+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(77 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 [stat,call,call] v31'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.030+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:78 3.16 3:689a7301:::2f19b8ff-2608-4d86-8f1c-430623205476.4231.6_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92400 con 0x55bc0be47000 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 16 ==== osd_op_reply(78 2f19b8ff-2608-4d86-8f1c-430623205476.4231.6_15f6f8dc036519d7fe15b39338f6e5db [call] v31'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:79 4.29 4:967bb226:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92800 con 0x55bc0be47800 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 20 list_objects_ordered: starting attempt 1 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:80 4.3c 4:3ff255e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92000 con 0x55bc0c8bd000 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:81 4.a 4:507ee1fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68c00 con 0x55bc0be47000 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:82 4.2d 4:b4b4270b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b000 con 0x55bc0be47c00 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:83 4.7 4:e3c21a1d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a800 con 0x55bc0be47c00 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:84 4.29 4:967bb226:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014000 con 0x55bc0be47800 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:85 4.10 4:08b73cb8:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014400 con 0x55bc0be47000 2024-04-27T05:23:20.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:86 4.a 4:53835284:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014800 con 0x55bc0be47000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 17 ==== osd_op_reply(81 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:87 4.2f 4:f74f9645:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== osd_op_reply(80 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:88 4.15 4:a95d8a74:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015000 con 0x55bc0be47800 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:89 4.5 4:a38b0747:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015400 con 0x55bc0be47800 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:90 4.f 4:f1d029e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015800 con 0x55bc0c8bd000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 18 ==== osd_op_reply(85 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.031+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(83 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(82 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 19 ==== osd_op_reply(86 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 21 ==== osd_op_reply(88 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 22 ==== osd_op_reply(89 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(87 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(90 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 23 ==== osd_op_reply(79 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 [stat,call,call] v31'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7c8bc9640 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 24 ==== osd_op_reply(84 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.6])"; uploads.size()=0; is_truncated=0 2024-04-27T05:23:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.6]) 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 20 lifecycle: rados->read ofs=0 len=0 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.032+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:91 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015c00 con 0x55bc0be47000 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.033+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 20 ==== osd_op_reply(91 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv39 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.033+0000 7fd7d1b26880 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.033+0000 7fd7d1b26880 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.033+0000 7fd7d1b26880 10 lifecycle: adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T05:23:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.033+0000 7fd7d1b26880 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.033+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:92 8.11 8:8d002013:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b400 con 0x55bc0be47800 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 25 ==== osd_op_reply(92 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 [delete,create,call,call,writefull 0~352,setxattr (123)] v31'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7d1b26880 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 info.flags=0x17 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7d1b26880 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 to cache LRU end 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7d1b26880 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7d1b26880 10 lifecycle: distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6, ofs0, ns] 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:93 7.15 7:a93a5511:::notify.2:head [notify cookie 94266141055616 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a800 con 0x55bc0be47800 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 26 ==== watch-notify(notify (1) cookie 94266141045248 notify 133143986197 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55bc0c9c4d00 con 0x55bc0be47800 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7c8bc9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986197 cookie 94266141045248 notifier 4861 bl.length()=725 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 27 ==== osd_op_reply(93 notify.2 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7c8bc9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 info.flags=0x17 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7c8bc9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 to cache LRU end 2024-04-27T05:23:20.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7c8bc9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.034+0000 7fd7c8bc9640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:94 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93000 con 0x55bc0be47800 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 28 ==== osd_op_reply(94 notify.2 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 29 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986197 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0c9c4ea0 con 0x55bc0be47800 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7d1b26880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.6]): orig_sources= new_sources= 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7d1b26880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.6]): potential sources added=[] removed=[] 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7d1b26880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.6]): orig_dests= new_dests= 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7d1b26880 20 lifecycle: handle_bi_update(): bucket=:myfoo[2f19b8ff-2608-4d86-8f1c-430623205476.4231.6]): potential dests added=[] removed=[] 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.035+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:33 6.11 6:8f661e56:lc::lc.12:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92c00 con 0x55bc0bf0ac00 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.036+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(33 lc.12 [call] v31'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.036+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 6.11 6:8f661e56:lc::lc.12:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93400 con 0x55bc0bf0ac00 2024-04-27T05:23:20.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(34 lc.12 [call] v31'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:35 6.11 6:8f661e56:lc::lc.12:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93800 con 0x55bc0bf0ac00 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7cebd5640 1 -- 172.21.15.138:0/2637311291 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(35 lc.12 [call] v31'39 uv39 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0bf0ac00 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:95 4.3c 4:3ff255e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92000 con 0x55bc0c8bd000 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:96 4.a 4:507ee1fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0be47000 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:97 4.2d 4:b4b4270b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a000 con 0x55bc0be47c00 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.037+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:98 4.7 4:e3c21a1d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0bc00 con 0x55bc0be47c00 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:99 4.29 4:967bb226:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b000 con 0x55bc0be47800 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:100 4.10 4:08b73cb8:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014000 con 0x55bc0be47000 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:101 4.a 4:53835284:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015000 con 0x55bc0be47000 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(95 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 21 ==== osd_op_reply(96 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:102 4.2f 4:f74f9645:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015400 con 0x55bc0c8bd000 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 30 ==== osd_op_reply(99 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 22 ==== osd_op_reply(100 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:103 4.15 4:a95d8a74:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93c00 con 0x55bc0be47800 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:104 4.5 4:a38b0747:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92800 con 0x55bc0be47800 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(97 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 22 ==== osd_op_reply(98 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 23 ==== osd_op_reply(101 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(102 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 31 ==== osd_op_reply(103 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 32 ==== osd_op_reply(104 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:105 4.f 4:f1d029e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69000 con 0x55bc0c8bd000 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 30 ==== osd_op_reply(105 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.038+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:106 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014400 con 0x55bc0be47000 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.040+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 24 ==== osd_op_reply(106 foo.buckets [call] v31'44 uv44 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.040+0000 7fd7d1b26880 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.040+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:107 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014800 con 0x55bc0be47000 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 25 ==== osd_op_reply(107 myfoo [call,call,delete] v31'45 uv45 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7d1b26880 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7d1b26880 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94266141055616 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015c00 con 0x55bc0be47c00 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 23 ==== watch-notify(notify (1) cookie 94266141049856 notify 133143986218 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x55bc0bebfa00 con 0x55bc0be47c00 2024-04-27T05:23:20.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7c8bc9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986218 cookie 94266141049856 notifier 4861 bl.length()=149 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7c8bc9640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92c00 con 0x55bc0be47c00 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 24 ==== osd_op_reply(108 notify.6 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 25 ==== osd_op_reply(109 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 26 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986218 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0bebfba0 con 0x55bc0be47c00 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.041+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:110 8.11 8:8d002013:root::.bucket.meta.myfoo%3a2f19b8ff-2608-4d86-8f1c-430623205476.4231.6:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014000 con 0x55bc0be47800 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.042+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 33 ==== osd_op_reply(110 .bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 [call,call,delete] v31'3 uv3 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.042+0000 7fd7d1b26880 10 removing default.rgw.meta+root+.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 from cache 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.042+0000 7fd7d1b26880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:2f19b8ff-2608-4d86-8f1c-430623205476.4231.6, ofs0, ns] 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.042+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:111 7.15 7:a93a5511:::notify.2:head [notify cookie 94266141055616 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0be47800 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 34 ==== watch-notify(notify (1) cookie 94266141045248 notify 133143986198 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x55bc0c9c4ea0 con 0x55bc0be47800 2024-04-27T05:23:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7c83c8640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986198 cookie 94266141045248 notifier 4861 bl.length()=206 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7c83c8640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:112 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93800 con 0x55bc0be47800 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 35 ==== osd_op_reply(111 notify.2 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 36 ==== osd_op_reply(112 notify.2 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 37 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986198 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0c9c5040 con 0x55bc0be47800 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:113 4.3c 4:3ff255e7:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68400 con 0x55bc0c8bd000 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:114 4.a 4:507ee1fd:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be68c00 con 0x55bc0be47000 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:115 4.2d 4:b4b4270b:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92400 con 0x55bc0be47c00 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:116 4.7 4:e3c21a1d:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014c00 con 0x55bc0be47c00 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:117 4.29 4:967bb226:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014000 con 0x55bc0be47800 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:118 4.10 4:08b73cb8:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015800 con 0x55bc0be47000 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:119 4.a 4:53835284:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015400 con 0x55bc0be47000 2024-04-27T05:23:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:120 4.2f 4:f74f9645:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:121 4.15 4:a95d8a74:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0be69000 con 0x55bc0be47800 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:122 4.5 4:a38b0747:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b400 con 0x55bc0be47800 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.043+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:123 4.f 4:f1d029e1:::.dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a800 con 0x55bc0c8bd000 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.044+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 31 ==== osd_op_reply(113 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.0 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 38 ==== osd_op_reply(121 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.8 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 27 ==== osd_op_reply(115 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.2 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 26 ==== osd_op_reply(114 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.1 [delete] v31'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 27 ==== osd_op_reply(118 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.5 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 28 ==== osd_op_reply(116 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.3 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 39 ==== osd_op_reply(117 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.4 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 40 ==== osd_op_reply(122 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.9 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 32 ==== osd_op_reply(120 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.7 [delete] v31'12 uv12 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.045+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 28 ==== osd_op_reply(119 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.6 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 33 ==== osd_op_reply(123 .dir.2f19b8ff-2608-4d86-8f1c-430623205476.4231.6.10 [delete] v31'6 uv6 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 : miss 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:124 6.b 6:d5f329a4:::pubsub..bucket.myfoo%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.6:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0b000 con 0x55bc0be47800 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 41 ==== osd_op_reply(124 pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 info.flags=0x0 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 10 adding default.rgw.log++pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 to cache LRU end 2024-04-27T05:23:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.046+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:125 6.b 6:d5f329a4:::pubsub..bucket.myfoo%2f2f19b8ff-2608-4d86-8f1c-430623205476.4231.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0a000 con 0x55bc0be47800 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.047+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 42 ==== osd_op_reply(125 pubsub..bucket.myfoo/2f19b8ff-2608-4d86-8f1c-430623205476.4231.6 [delete] v31'35 uv27 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.047+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:126 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0bc00 con 0x55bc0be47000 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.048+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 29 ==== osd_op_reply(126 foo.buckets [call] v31'46 uv46 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.048+0000 7fd7d1b26880 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.048+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:127 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93000 con 0x55bc0be47c00 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 29 ==== osd_op_reply(127 9te6NH5mcdcq0Tc5i8i1 [delete] v31'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7d1b26880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:128 7.e 7:7759931f:::notify.3:head [notify cookie 94266141055616 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93c00 con 0x55bc0be47000 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 30 ==== watch-notify(notify (1) cookie 94266141046400 notify 133143986196 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55bc0bebfd40 con 0x55bc0be47000 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7c83c8640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986196 cookie 94266141046400 notifier 4861 bl.length()=170 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 31 ==== osd_op_reply(128 notify.3 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.049+0000 7fd7c83c8640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:129 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfac000 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.050+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 32 ==== osd_op_reply(129 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.050+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 33 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986196 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0bebd860 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.050+0000 7fd7d1b26880 10 removing email index: foo@foo.com 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.050+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:130 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92800 con 0x55bc0c8bd000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 34 ==== osd_op_reply(130 foo@foo.com [delete] v31'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7d1b26880 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:131 7.e 7:7759931f:::notify.3:head [notify cookie 94266141055616 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93800 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 34 ==== watch-notify(notify (1) cookie 94266141046400 notify 133143986197 ret 0) v3 ==== 204+0+0 (crc 0 0 0) 0x55bc0bebd860 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7c83c8640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986197 cookie 94266141046400 notifier 4861 bl.length()=162 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7c83c8640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:132 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfac400 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 35 ==== osd_op_reply(131 notify.3 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 36 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986197 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0bebaea0 con 0x55bc0be47000 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 37 ==== osd_op_reply(132 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7d1b26880 10 removing user buckets index 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.051+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:133 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf92000 con 0x55bc0be47000 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 38 ==== osd_op_reply(133 foo.buckets [delete] v31'47 uv47 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7d1b26880 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94266141055616 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c8bd400 con 0x55bc0be47c00 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 30 ==== watch-notify(notify (1) cookie 94266141048704 notify 133143986219 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55bc0bebfba0 con 0x55bc0be47c00 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7c8bc9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986219 cookie 94266141048704 notifier 4861 bl.length()=160 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7c8bc9640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93400 con 0x55bc0be47c00 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.052+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 31 ==== osd_op_reply(134 notify.5 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.053+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 32 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.053+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 33 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986219 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0bebe1a0 con 0x55bc0be47c00 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.053+0000 7fd7d1b26880 10 removing user index: foo 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.053+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:136 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfac000 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 35 ==== osd_op_reply(136 foo [call,call,delete] v31'20 uv20 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7d1b26880 10 removing default.rgw.meta+users.uid+foo from cache 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7d1b26880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:137 7.d 7:b4812045:::notify.4:head [notify cookie 94266141055616 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfac800 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 36 ==== watch-notify(notify (1) cookie 94266141047552 notify 133143986196 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55bc0bebe1a0 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7c83c8640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986196 cookie 94266141047552 notifier 4861 bl.length()=152 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7c83c8640 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 37 ==== osd_op_reply(137 notify.4 [notify cookie 94266141055616 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 38 ==== osd_op_reply(138 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.054+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 39 ==== watch-notify(notify_complete (2) cookie 94266141055616 notify 133143986196 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55bc0bebe340 con 0x55bc0c8bd000 2024-04-27T05:23:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.055+0000 7fd7b53a2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.055+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:139 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94266141052160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfacc00 con 0x55bc0c8bd000 2024-04-27T05:23:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.056+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 40 ==== osd_op_reply(139 data_loggenerations_metadata [watch unwatch cookie 94266141052160] v31'242 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.056+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:140 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94266141042944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfad000 con 0x55bc0be47000 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.057+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 39 ==== osd_op_reply(140 notify.0 [watch unwatch cookie 94266141042944] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.057+0000 7fd7d1b26880 20 remove_watcher() i=0 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.057+0000 7fd7d1b26880 2 removed watcher, disabling cache 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.057+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94266141044096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfad400 con 0x55bc0c8bd000 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.058+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 41 ==== osd_op_reply(141 notify.1 [watch unwatch cookie 94266141044096] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.058+0000 7fd7d1b26880 20 remove_watcher() i=1 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.058+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:142 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94266141045248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfad800 con 0x55bc0be47800 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.059+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/1492183943 <== osd.3 v2:172.21.15.138:6808/29866 43 ==== osd_op_reply(142 notify.2 [watch unwatch cookie 94266141045248] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5afc0 con 0x55bc0be47800 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.059+0000 7fd7d1b26880 20 remove_watcher() i=2 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.059+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:143 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94266141046400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bfadc00 con 0x55bc0be47000 2024-04-27T05:23:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.060+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 <== osd.2 v2:172.21.15.138:6809/29865 40 ==== osd_op_reply(143 notify.3 [watch unwatch cookie 94266141046400] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5b200 con 0x55bc0be47000 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.060+0000 7fd7d1b26880 20 remove_watcher() i=3 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.060+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:144 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94266141047552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf93c00 con 0x55bc0c8bd000 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.061+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.1 v2:172.21.15.138:6800/29864 42 ==== osd_op_reply(144 notify.4 [watch unwatch cookie 94266141047552] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0c8bd000 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.061+0000 7fd7d1b26880 20 remove_watcher() i=4 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.061+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:145 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94266141048704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0bf0bc00 con 0x55bc0be47c00 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.062+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 34 ==== osd_op_reply(145 notify.5 [watch unwatch cookie 94266141048704] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.062+0000 7fd7d1b26880 20 remove_watcher() i=5 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.062+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:146 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94266141049856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c015000 con 0x55bc0be47c00 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.063+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 35 ==== osd_op_reply(146 notify.6 [watch unwatch cookie 94266141049856] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.063+0000 7fd7d1b26880 20 remove_watcher() i=6 2024-04-27T05:23:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.063+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:147 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94266141051008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bc0c014400 con 0x55bc0be47c00 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7cebd5640 1 -- 172.21.15.138:0/1492183943 <== osd.0 v2:172.21.15.114:6800/27747 36 ==== osd_op_reply(147 notify.7 [watch unwatch cookie 94266141051008] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc0be5a900 con 0x55bc0be47c00 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 20 remove_watcher() i=7 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0be47c00 msgr2=0x55bc0bda9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0be47c00 0x55bc0bda9180 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0c8bd000 msgr2=0x55bc0bda7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0c8bd000 0x55bc0bda7b80 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0be47000 msgr2=0x55bc0bda8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0be47000 0x55bc0bda8c00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0be47800 msgr2=0x55bc0bda8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.064+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0be47800 0x55bc0bda8100 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0be46000 msgr2=0x55bc0bda7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0be46000 0x55bc0bda7600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55bc0bbd1380 tx=0x55bc0ca22f60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 msgr2=0x55bc0bda6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/1492183943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0c8bcc00 0x55bc0bda6b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55bc0c980b40 tx=0x55bc0bbd7b00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 reap_dead start 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 shutdown_connections 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 >> 172.21.15.138:0/1492183943 conn(0x55bc0bac5400 msgr2=0x55bc0bbcf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7cdbd3640 1 -- 172.21.15.138:0/1492183943 reap_dead start 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 shutdown_connections 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/1492183943 wait complete. 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0bf0a400 msgr2=0x55bc0bf06580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bc0bf0a400 0x55bc0bf06580 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0ca24800 msgr2=0x55bc0bf06000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bc0ca24800 0x55bc0bf06000 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0bf0b800 msgr2=0x55bc0bf07080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bc0bf0b800 0x55bc0bf07080 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0bf0ac00 msgr2=0x55bc0bf06b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bc0bf0ac00 0x55bc0bf06b00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0ca24c00 msgr2=0x55bc0bda9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0ca24c00 0x55bc0bda9700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55bc0c9c6f90 tx=0x55bc0ca23e60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 msgr2=0x55bc0bda8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55bc0be25800 tx=0x55bc0ca23080 comp rx=0 tx=0).stop 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7ce3d4640 1 -- 172.21.15.138:0/2637311291 reap_dead start 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 shutdown_connections 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2637311291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0be46c00 0x55bc0bda8680 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 >> 172.21.15.138:0/2637311291 conn(0x55bc0be69800 msgr2=0x55bc0bbcf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 shutdown_connections 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.065+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2637311291 wait complete. 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0c8bdc00 msgr2=0x55bc0bda7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0c8bdc00 0x55bc0bda7080 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55bc0c9c6180 tx=0x55bc0bbd7c20 comp rx=0 tx=0).stop 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 msgr2=0x55bc0bda6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x55bc0c9812c0 tx=0x55bc0bbd6c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 shutdown_connections 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bc0c8bdc00 0x55bc0bda7080 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 --2- 172.21.15.138:0/2372280444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bc0bac5000 0x55bc0bda6580 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 >> 172.21.15.138:0/2372280444 conn(0x55bc0bac4c00 msgr2=0x55bc0bbcd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 shutdown_connections 2024-04-27T05:23:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.066+0000 7fd7d1b26880 1 -- 172.21.15.138:0/2372280444 wait complete. 2024-04-27T05:23:20.105 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T05:23:20.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T05:23:20.105 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T05:23:20.181 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.181 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12f146880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12f146880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d3463be600 con 0x55d3460e7400 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12f146880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d346126280 con 0x55d3460e7000 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12d8eb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12d0ea640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55d3460e7400 0x55d3461ee800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38266/0 (socket says 172.21.15.138:38266) 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12d8eb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35326/0 (socket says 172.21.15.138:35326) 2024-04-27T05:23:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12d8eb640 1 -- 172.21.15.138:0/2786806994 learned_addr learned my addr 172.21.15.138:0/2786806994 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12d8eb640 1 -- 172.21.15.138:0/2786806994 >> v1:172.21.15.114:6789/0 conn(0x55d3460e7400 legacy=0x55d3461ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.180+0000 7fa12d8eb640 1 -- 172.21.15.138:0/2786806994 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3460eb380 con 0x55d3460e7000 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.182+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/2786806994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x55d3461f2d50 tx=0x55d3461f8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30899f087464c09 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.182+0000 7fa12c8e9640 1 -- 172.21.15.138:0/2786806994 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ffce0 con 0x55d3460e7000 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.182+0000 7fa12c8e9640 1 -- 172.21.15.138:0/2786806994 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3463bfb00 con 0x55d3460e7000 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.182+0000 7fa12c8e9640 1 -- 172.21.15.138:0/2786806994 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ffe40 con 0x55d3460e7000 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 -- 172.21.15.138:0/2786806994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 msgr2=0x55d3463c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 --2- 172.21.15.138:0/2786806994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x55d3461f2d50 tx=0x55d3461f8de0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 -- 172.21.15.138:0/2786806994 shutdown_connections 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 --2- 172.21.15.138:0/2786806994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 -- 172.21.15.138:0/2786806994 >> 172.21.15.138:0/2786806994 conn(0x55d3460e6c00 msgr2=0x55d3461ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 -- 172.21.15.138:0/2786806994 shutdown_connections 2024-04-27T05:23:20.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 -- 172.21.15.138:0/2786806994 wait complete. 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.183+0000 7fa12f146880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12f146880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d346126780 con 0x55d3460e7000 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12d8eb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12d8eb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35342/0 (socket says 172.21.15.138:35342) 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12d8eb640 1 -- 172.21.15.138:0/183677264 learned_addr learned my addr 172.21.15.138:0/183677264 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12d8eb640 1 -- 172.21.15.138:0/183677264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3460eaea0 con 0x55d3460e7000 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/183677264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x55d346eeb560 tx=0x55d3461f8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87206e9ea7fd6b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12c0e8640 1 -- 172.21.15.138:0/183677264 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ff4a0 con 0x55d3460e7000 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12c0e8640 1 -- 172.21.15.138:0/183677264 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d3463bfe00 con 0x55d3460e7000 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.184+0000 7fa12c0e8640 1 -- 172.21.15.138:0/183677264 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ffce0 con 0x55d3460e7000 2024-04-27T05:23:20.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- 172.21.15.138:0/183677264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 msgr2=0x55d3463c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 --2- 172.21.15.138:0/183677264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x55d346eeb560 tx=0x55d3461f8cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- 172.21.15.138:0/183677264 shutdown_connections 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 --2- 172.21.15.138:0/183677264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- 172.21.15.138:0/183677264 >> 172.21.15.138:0/183677264 conn(0x55d3460e6c00 msgr2=0x55d3461ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- 172.21.15.138:0/183677264 shutdown_connections 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- 172.21.15.138:0/183677264 wait complete. 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12f146880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3461268c0 con 0x55d3460e7000 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12d8eb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12d8eb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35356/0 (socket says 172.21.15.138:35356) 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.185+0000 7fa12d8eb640 1 -- 172.21.15.138:0/3053191305 learned_addr learned my addr 172.21.15.138:0/3053191305 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12d8eb640 1 -- 172.21.15.138:0/3053191305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3460eb6c0 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55d346fa32c0 tx=0x55d3461f8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12a8e5640 1 -- 172.21.15.138:0/3053191305 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ffe40 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12a8e5640 1 -- 172.21.15.138:0/3053191305 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d346ec6000 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3460eb1e0 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12a8e5640 1 -- 172.21.15.138:0/3053191305 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ff4a0 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d346158ea0 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.186+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12a8e5640 1 -- 172.21.15.138:0/3053191305 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d346110b00 con 0x55d3460e7000 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12f146880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12f146880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d346126c80 con 0x55d346edec00 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12a8e5640 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346edfc00 0x55d3463c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12a8e5640 1 -- 172.21.15.138:0/3053191305 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55d3460f9a40 con 0x55d3460e7000 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12e0ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346edfc00 0x55d3463c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12e0ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35372/0 (socket says 172.21.15.138:35372) 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12e0ec640 1 -- 172.21.15.138:0/2137317796 learned_addr learned my addr 172.21.15.138:0/2137317796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12e0ec640 1 -- 172.21.15.138:0/2137317796 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d346158d00 con 0x55d346edec00 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346edfc00 0x55d3463c9080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d346fe8780 tx=0x55d3461f9c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/2137317796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55d346fa3680 tx=0x55d3461f9f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c9b6e30ee7cf4d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa1280e0640 1 -- 172.21.15.138:0/2137317796 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ffe40 con 0x55d346edec00 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa1280e0640 1 -- 172.21.15.138:0/2137317796 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d346ec6600 con 0x55d346edec00 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.187+0000 7fa1280e0640 1 -- 172.21.15.138:0/2137317796 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ff4a0 con 0x55d346edec00 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- 172.21.15.138:0/2137317796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 msgr2=0x55d3463c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 --2- 172.21.15.138:0/2137317796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55d346fa3680 tx=0x55d3461f9f80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- 172.21.15.138:0/2137317796 shutdown_connections 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 --2- 172.21.15.138:0/2137317796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- 172.21.15.138:0/2137317796 >> 172.21.15.138:0/2137317796 conn(0x55d3460e7400 msgr2=0x55d3461f1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- 172.21.15.138:0/2137317796 shutdown_connections 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- 172.21.15.138:0/2137317796 wait complete. 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12f146880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34610a280 con 0x55d346edec00 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12e0ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12e0ec640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35386/0 (socket says 172.21.15.138:35386) 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.188+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 learned_addr learned my addr 172.21.15.138:0/743300716 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d346158b60 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55d346fa33b0 tx=0x55d3461f9b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa1268dd640 1 -- 172.21.15.138:0/743300716 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ffe40 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa1268dd640 1 -- 172.21.15.138:0/743300716 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d346ec6780 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3461589c0 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa1268dd640 1 -- 172.21.15.138:0/743300716 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3460ff4a0 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.189+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d346158820 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa1268dd640 1 -- 172.21.15.138:0/743300716 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d346111600 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa1268dd640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346466000 0x55d3463c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa1268dd640 1 -- 172.21.15.138:0/743300716 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55d3460f9c00 con 0x55d346edec00 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d346edf000 0x55d3463c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346466000 0x55d3463c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346edf400 con 0x55d346edf000 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d346edf000 0x55d3463c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346466000 0x55d3463c9600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55d3461f3380 tx=0x55d347042f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.190+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d346edf000 0x55d3463c9b80 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 realm 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346edf800 con 0x55d346edf000 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:23:20.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467000 con 0x55d346edf000 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467400 con 0x55d346edf000 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d346467800 0x55d3463ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467c00 con 0x55d346467800 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.191+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d346467800 0x55d3463ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d346467800 0x55d3463ca100 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12f146880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12f146880 20 searching for the correct realm 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467000 con 0x55d346467800 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12d8eb640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d346467000 0x55d3463cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467c00 con 0x55d346467000 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.192+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d346467000 0x55d3463cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.193+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d346467000 0x55d3463cac00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.193+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.193+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d346467c00 0x55d3463cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.193+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.193+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d346467c00 0x55d3463cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.193+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d346467c00 0x55d3463cb180 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.194+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467000 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.195+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.195+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.195+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.195+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.195+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.195+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467000 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.196+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.197+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467000 2024-04-27T05:23:20.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.198+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467000 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.199+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467800 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467c00 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346edf000 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346466c00 con 0x55d346467000 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12f146880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12f146880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12f146880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12f146880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.200+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467400 con 0x55d346edf000 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346edf400 con 0x55d346edf000 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346edf800 con 0x55d346467800 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 zone default found 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 4 Realm: () 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:23:20.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 zonegroup default 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488000 con 0x55d346467c00 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.201+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488400 con 0x55d346467800 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488800 con 0x55d346467800 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 20 started sync module instance, tier type = 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.202+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488c00 con 0x55d346467000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.203+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.203+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94365904800000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489000 con 0x55d346467000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.203+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489400 con 0x55d346edf000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.204+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94365904800000] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.204+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.204+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94365904801152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489800 con 0x55d346edf000 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.204+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489c00 con 0x55d346467800 2024-04-27T05:23:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.205+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94365904801152] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.206+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.206+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94365904802304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488400 con 0x55d346467800 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.206+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488800 con 0x55d346467000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.207+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94365904802304] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.207+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.207+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94365904803456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489c00 con 0x55d346467000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.207+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488000 con 0x55d346edf000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.208+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94365904803456] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.208+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.208+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94365904804608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346edf800 con 0x55d346edf000 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.208+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346edf400 con 0x55d346467c00 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.209+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.209+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94365904805760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346467400 con 0x55d346467c00 2024-04-27T05:23:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.209+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d347044c00 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.209+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94365904804608] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.210+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94365904805760] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.210+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.210+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94365904806912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d347044800 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.210+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488c00 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(58 notify.7 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94365904806912] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94365904808064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489000 con 0x55d346467c00 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=0 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=1 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=2 2024-04-27T05:23:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=3 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=4 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=5 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.211+0000 7fa12f146880 20 add_watcher() i=6 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.212+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94365904808064] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.212+0000 7fa12f146880 20 add_watcher() i=7 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.212+0000 7fa12f146880 2 all 8 watchers are set, enabling cache 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.212+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346488800 con 0x55d346edf000 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.212+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.212+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94365904809216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d346489400 con 0x55d346edf000 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.213+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94365904809216] v31'243 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.213+0000 7fa1148b9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.213+0000 7fa12f146880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.213+0000 7fa12f146880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.213+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12f146880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12f146880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34610a8c0 con 0x55d346466c00 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12d0ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12d0ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35396/0 (socket says 172.21.15.138:35396) 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12d0ea640 1 -- 172.21.15.138:0/2626270034 learned_addr learned my addr 172.21.15.138:0/2626270034 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12d0ea640 1 -- 172.21.15.138:0/2626270034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d346158680 con 0x55d346466c00 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.214+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/2626270034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55d346446ae0 tx=0x55d347042f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bc4853b9709a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa1140b8640 1 -- 172.21.15.138:0/2626270034 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d34648a580 con 0x55d346466c00 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa1140b8640 1 -- 172.21.15.138:0/2626270034 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d346ec6000 con 0x55d346466c00 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa1140b8640 1 -- 172.21.15.138:0/2626270034 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d34648a6e0 con 0x55d346466c00 2024-04-27T05:23:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 -- 172.21.15.138:0/2626270034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 msgr2=0x55d3463ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 --2- 172.21.15.138:0/2626270034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55d346446ae0 tx=0x55d347042f00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 -- 172.21.15.138:0/2626270034 shutdown_connections 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 --2- 172.21.15.138:0/2626270034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 -- 172.21.15.138:0/2626270034 >> 172.21.15.138:0/2626270034 conn(0x55d346489800 msgr2=0x55d3461f1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 -- 172.21.15.138:0/2626270034 shutdown_connections 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.215+0000 7fa12f146880 1 -- 172.21.15.138:0/2626270034 wait complete. 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12f146880 1 Processor -- start 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12f146880 1 -- start start 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12f146880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12f146880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34610a780 con 0x55d346466c00 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12d0ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12d0ea640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35406/0 (socket says 172.21.15.138:35406) 2024-04-27T05:23:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12d0ea640 1 -- 172.21.15.138:0/3695894887 learned_addr learned my addr 172.21.15.138:0/3695894887 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12d0ea640 1 -- 172.21.15.138:0/3695894887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3461584e0 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa12d0ea640 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55d346447860 tx=0x55d347043080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa1128b5640 1 -- 172.21.15.138:0/3695894887 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d34648a840 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa1128b5640 1 -- 172.21.15.138:0/3695894887 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d346ec6a80 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.216+0000 7fa1128b5640 1 -- 172.21.15.138:0/3695894887 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d34648a580 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3460eb380 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3460eaea0 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa1128b5640 1 -- 172.21.15.138:0/3695894887 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55d346111b80 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa1128b5640 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d347044c00 0x55d3463cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa1128b5640 1 -- 172.21.15.138:0/3695894887 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55d3460f9dc0 con 0x55d346466c00 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa12f146880 5 note: GC not initialized 2024-04-27T05:23:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.217+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d347044c00 0x55d3463cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa1100b0640 20 reqs_thread_entry: start 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12f146880 20 init_complete bucket index max shards: 11 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12e0ec640 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d347044c00 0x55d3463cb700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55d347050c30 tx=0x55d347043e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12f146880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa10e0ac640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:50 2024 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa10d88a640 20 reqs_thread_entry: start 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12f146880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655c000 con 0x55d346edf000 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.218+0000 7fa12f146880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:could not fetch user info: no user info saved 2024-04-27T05:23:20.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.219+0000 7fa1148b9640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.219+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94365904809216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655c400 con 0x55d346edf000 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.220+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94365904809216] v31'244 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.220+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94365904800000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655c800 con 0x55d346467000 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.221+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94365904800000] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.221+0000 7fa12f146880 20 remove_watcher() i=0 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.221+0000 7fa12f146880 2 removed watcher, disabling cache 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.221+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94365904801152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655cc00 con 0x55d346edf000 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.222+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94365904801152] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.222+0000 7fa12f146880 20 remove_watcher() i=1 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.222+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94365904802304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655d000 con 0x55d346467800 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.223+0000 7fa12d8eb640 1 -- 172.21.15.138:0/743300716 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94365904802304] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647afc0 con 0x55d346467800 2024-04-27T05:23:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.223+0000 7fa12f146880 20 remove_watcher() i=2 2024-04-27T05:23:20.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.223+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94365904803456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655d400 con 0x55d346467000 2024-04-27T05:23:20.245 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-27T05:23:20.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.224+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94365904803456] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647b200 con 0x55d346467000 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.224+0000 7fa12f146880 20 remove_watcher() i=3 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.224+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94365904804608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655d800 con 0x55d346edf000 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.225+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94365904804608] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346edf000 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.225+0000 7fa12f146880 20 remove_watcher() i=4 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.225+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94365904805760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655dc00 con 0x55d346467c00 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.226+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94365904805760] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.226+0000 7fa12f146880 20 remove_watcher() i=5 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.226+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94365904806912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655c000 con 0x55d346467c00 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.227+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94365904806912] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.227+0000 7fa12f146880 20 remove_watcher() i=6 2024-04-27T05:23:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.227+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94365904808064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d34655c400 con 0x55d346467c00 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12e0ec640 1 -- 172.21.15.138:0/743300716 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94365904808064] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d34647a900 con 0x55d346467c00 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 20 remove_watcher() i=7 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d346467c00 msgr2=0x55d3463cb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55d346467c00 0x55d3463cb180 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d346edf000 msgr2=0x55d3463c9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55d346edf000 0x55d3463c9b80 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d346467000 msgr2=0x55d3463cac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55d346467000 0x55d3463cac00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d346467800 msgr2=0x55d3463ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55d346467800 0x55d3463ca100 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346466000 msgr2=0x55d3463c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346466000 0x55d3463c9600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55d3461f3380 tx=0x55d347042f60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 msgr2=0x55d3463c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 --2- 172.21.15.138:0/743300716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346edec00 0x55d3463c8b00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55d346fa33b0 tx=0x55d3461f9b00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 reap_dead start 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 shutdown_connections 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 >> 172.21.15.138:0/743300716 conn(0x55d3460e7400 msgr2=0x55d3461f1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12d0ea640 1 -- 172.21.15.138:0/743300716 reap_dead start 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 shutdown_connections 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.228+0000 7fa12f146880 1 -- 172.21.15.138:0/743300716 wait complete. 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d347044c00 msgr2=0x55d3463cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d347044c00 0x55d3463cb700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55d347050c30 tx=0x55d347043e60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 msgr2=0x55d3463ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55d346447860 tx=0x55d347043080 comp rx=0 tx=0).stop 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 shutdown_connections 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d347044c00 0x55d3463cb700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3695894887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d346466c00 0x55d3463ca680 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 >> 172.21.15.138:0/3695894887 conn(0x55d346489800 msgr2=0x55d3461f1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 shutdown_connections 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3695894887 wait complete. 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346edfc00 msgr2=0x55d3463c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346edfc00 0x55d3463c9080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d346fe8780 tx=0x55d3461f9c20 comp rx=0 tx=0).stop 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 msgr2=0x55d3463c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55d346fa32c0 tx=0x55d3461f8c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 shutdown_connections 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55d346edfc00 0x55d3463c9080 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 --2- 172.21.15.138:0/3053191305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d3460e7000 0x55d3463c8580 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 >> 172.21.15.138:0/3053191305 conn(0x55d3460e6c00 msgr2=0x55d3461ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 shutdown_connections 2024-04-27T05:23:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.229+0000 7fa12f146880 1 -- 172.21.15.138:0/3053191305 wait complete. 2024-04-27T05:23:20.249 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-04-27T05:23:20.250 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-04-27T05:23:20.250 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-04-27T05:23:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50fca19880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55972ed69000 0x55972f04a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50fca19880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55972f040600 con 0x55972ed69400 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50fca19880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55972eda8280 con 0x55972ed69000 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50f8efb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55972ed69000 0x55972f04a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50f8efb640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55972ed69000 0x55972f04a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35420/0 (socket says 172.21.15.138:35420) 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50f8efb640 1 -- 172.21.15.138:0/3948577913 learned_addr learned my addr 172.21.15.138:0/3948577913 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50f86fa640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55972ed69400 0x55972ee70800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38272/0 (socket says 172.21.15.138:38272) 2024-04-27T05:23:20.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.326+0000 7f50f8efb640 1 -- 172.21.15.138:0/3948577913 >> v1:172.21.15.114:6789/0 conn(0x55972ed69400 legacy=0x55972ee70800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.327+0000 7f50f8efb640 1 -- 172.21.15.138:0/3948577913 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972ed6d380 con 0x55972ed69000 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.328+0000 7f50f8efb640 1 --2- 172.21.15.138:0/3948577913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69000 0x55972f04a580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x55972ee74d50 tx=0x55972ee7ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458ee0b33849dab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.328+0000 7f50f7ef9640 1 -- 172.21.15.138:0/3948577913 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81ce0 con 0x55972ed69000 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.328+0000 7f50f7ef9640 1 -- 172.21.15.138:0/3948577913 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972f041b00 con 0x55972ed69000 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.328+0000 7f50f7ef9640 1 -- 172.21.15.138:0/3948577913 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81e40 con 0x55972ed69000 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 -- 172.21.15.138:0/3948577913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69000 msgr2=0x55972f04a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 --2- 172.21.15.138:0/3948577913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69000 0x55972f04a580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x55972ee74d50 tx=0x55972ee7ade0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 -- 172.21.15.138:0/3948577913 shutdown_connections 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 --2- 172.21.15.138:0/3948577913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69000 0x55972f04a580 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 -- 172.21.15.138:0/3948577913 >> 172.21.15.138:0/3948577913 conn(0x55972ed68c00 msgr2=0x55972ee70000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 -- 172.21.15.138:0/3948577913 shutdown_connections 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 -- 172.21.15.138:0/3948577913 wait complete. 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.329+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50fca19880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50fca19880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972eda8780 con 0x55972ed69400 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35432/0 (socket says 172.21.15.138:35432) 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f8efb640 1 -- 172.21.15.138:0/2356418812 learned_addr learned my addr 172.21.15.138:0/2356418812 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f8efb640 1 -- 172.21.15.138:0/2356418812 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972ed6cea0 con 0x55972ed69400 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f8efb640 1 --2- 172.21.15.138:0/2356418812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55972ee74d50 tx=0x55972ee7acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e272aa1f2236c460 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f76f8640 1 -- 172.21.15.138:0/2356418812 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed814a0 con 0x55972ed69400 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f76f8640 1 -- 172.21.15.138:0/2356418812 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972f041e00 con 0x55972ed69400 2024-04-27T05:23:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.330+0000 7f50f76f8640 1 -- 172.21.15.138:0/2356418812 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81ce0 con 0x55972ed69400 2024-04-27T05:23:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- 172.21.15.138:0/2356418812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 msgr2=0x55972f04ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 --2- 172.21.15.138:0/2356418812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55972ee74d50 tx=0x55972ee7acc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- 172.21.15.138:0/2356418812 shutdown_connections 2024-04-27T05:23:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 --2- 172.21.15.138:0/2356418812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- 172.21.15.138:0/2356418812 >> 172.21.15.138:0/2356418812 conn(0x55972ed68c00 msgr2=0x55972ee71000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- 172.21.15.138:0/2356418812 shutdown_connections 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- 172.21.15.138:0/2356418812 wait complete. 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50fca19880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972eda88c0 con 0x55972ed69400 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35440/0 (socket says 172.21.15.138:35440) 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.331+0000 7f50f8efb640 1 -- 172.21.15.138:0/764587690 learned_addr learned my addr 172.21.15.138:0/764587690 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50f8efb640 1 -- 172.21.15.138:0/764587690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972ed6d6c0 con 0x55972ed69400 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50f8efb640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x55972fc24c00 tx=0x55972ee7ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50f5ef5640 1 -- 172.21.15.138:0/764587690 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81e40 con 0x55972ed69400 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50f5ef5640 1 -- 172.21.15.138:0/764587690 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972fb48000 con 0x55972ed69400 2024-04-27T05:23:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55972ed6d1e0 con 0x55972ed69400 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50f5ef5640 1 -- 172.21.15.138:0/764587690 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed814a0 con 0x55972ed69400 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.332+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55972eddaea0 con 0x55972ed69400 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50fca19880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50fca19880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972eda8c80 con 0x55972fb5b000 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f5ef5640 1 -- 172.21.15.138:0/764587690 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55972ed92b00 con 0x55972ed69400 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f96fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f96fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35456/0 (socket says 172.21.15.138:35456) 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f96fc640 1 -- 172.21.15.138:0/2824315205 learned_addr learned my addr 172.21.15.138:0/2824315205 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f5ef5640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972fb5bc00 0x55972f04c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f5ef5640 1 -- 172.21.15.138:0/764587690 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55972ed7ba40 con 0x55972ed69400 2024-04-27T05:23:20.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f8efb640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972fb5bc00 0x55972f04c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f96fc640 1 -- 172.21.15.138:0/2824315205 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972eddad00 con 0x55972fb5b000 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f96fc640 1 --2- 172.21.15.138:0/2824315205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55972fc53a70 tx=0x55972ee7bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4800b9d2af4dbecd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f8efb640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972fb5bc00 0x55972f04c100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55972fc52660 tx=0x55972ee7bd40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f36f0640 1 -- 172.21.15.138:0/2824315205 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81e40 con 0x55972fb5b000 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.333+0000 7f50f36f0640 1 -- 172.21.15.138:0/2824315205 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972fb48600 con 0x55972fb5b000 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50f36f0640 1 -- 172.21.15.138:0/2824315205 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed814a0 con 0x55972fb5b000 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- 172.21.15.138:0/2824315205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 msgr2=0x55972f04b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 --2- 172.21.15.138:0/2824315205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55972fc53a70 tx=0x55972ee7bbc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- 172.21.15.138:0/2824315205 shutdown_connections 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 --2- 172.21.15.138:0/2824315205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- 172.21.15.138:0/2824315205 >> 172.21.15.138:0/2824315205 conn(0x55972fb5ac00 msgr2=0x55972ee73000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- 172.21.15.138:0/2824315205 shutdown_connections 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- 172.21.15.138:0/2824315205 wait complete. 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.334+0000 7f50fca19880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ed8c280 con 0x55972fb5b000 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f96fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f96fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35460/0 (socket says 172.21.15.138:35460) 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f96fc640 1 -- 172.21.15.138:0/354590495 learned_addr learned my addr 172.21.15.138:0/354590495 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f96fc640 1 -- 172.21.15.138:0/354590495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972eddab60 con 0x55972fb5b000 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f96fc640 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55972fc256b0 tx=0x55972ee7b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f1eed640 1 -- 172.21.15.138:0/354590495 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81e40 con 0x55972fb5b000 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f1eed640 1 -- 172.21.15.138:0/354590495 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972fb48780 con 0x55972fb5b000 2024-04-27T05:23:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55972edda9c0 con 0x55972fb5b000 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50f1eed640 1 -- 172.21.15.138:0/354590495 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed814a0 con 0x55972fb5b000 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.335+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55972edda820 con 0x55972fb5b000 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50f1eed640 1 -- 172.21.15.138:0/354590495 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55972ed93600 con 0x55972fb5b000 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50f1eed640 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f0e4000 0x55972f04c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50f1eed640 1 -- 172.21.15.138:0/354590495 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55972ed7bc00 con 0x55972fb5b000 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50f86fa640 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f0e4000 0x55972f04c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50fca19880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50fca19880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.336+0000 7f50f86fa640 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f0e4000 0x55972f04c680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55972ee75380 tx=0x55972fcd4d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50fca19880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50fca19880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ed8d180 con 0x55972fb5b800 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35464/0 (socket says 172.21.15.138:35464) 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50f8efb640 1 -- 172.21.15.138:0/1477768080 learned_addr learned my addr 172.21.15.138:0/1477768080 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50f8efb640 1 -- 172.21.15.138:0/1477768080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972edda680 con 0x55972fb5b800 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50f8efb640 1 --2- 172.21.15.138:0/1477768080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55972fc527e0 tx=0x55972ee7be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb7fea9a8810cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50dfec9640 1 -- 172.21.15.138:0/1477768080 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81ce0 con 0x55972fb5b800 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50dfec9640 1 -- 172.21.15.138:0/1477768080 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972fb48300 con 0x55972fb5b800 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.337+0000 7f50dfec9640 1 -- 172.21.15.138:0/1477768080 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81e40 con 0x55972fb5b800 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- 172.21.15.138:0/1477768080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 msgr2=0x55972f04b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 --2- 172.21.15.138:0/1477768080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55972fc527e0 tx=0x55972ee7be00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- 172.21.15.138:0/1477768080 shutdown_connections 2024-04-27T05:23:20.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 --2- 172.21.15.138:0/1477768080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- 172.21.15.138:0/1477768080 >> 172.21.15.138:0/1477768080 conn(0x55972fb5b400 msgr2=0x55972ee73800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- 172.21.15.138:0/1477768080 shutdown_connections 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- 172.21.15.138:0/1477768080 wait complete. 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 Processor -- start 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- start start 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50fca19880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972ed8cb40 con 0x55972fb5b800 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50f8efb640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35474/0 (socket says 172.21.15.138:35474) 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.338+0000 7f50f8efb640 1 -- 172.21.15.138:0/4189997590 learned_addr learned my addr 172.21.15.138:0/4189997590 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50f8efb640 1 -- 172.21.15.138:0/4189997590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55972edda4e0 con 0x55972fb5b800 2024-04-27T05:23:20.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50f8efb640 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x55972fc52ab0 tx=0x55972ee7bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50de6c6640 1 -- 172.21.15.138:0/4189997590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed814a0 con 0x55972fb5b800 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50de6c6640 1 -- 172.21.15.138:0/4189997590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55972fb48480 con 0x55972fb5b800 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55972ed6d380 con 0x55972fb5b800 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50de6c6640 1 -- 172.21.15.138:0/4189997590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55972ed81ce0 con 0x55972fb5b800 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55972ed6cea0 con 0x55972fb5b800 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50dc6c2640 20 reqs_thread_entry: start 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55972f0e5400 0x55972f04bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55972f0e5800 con 0x55972f0e5400 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50f86fa640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55972f0e5400 0x55972f04bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.339+0000 7f50de6c6640 1 -- 172.21.15.138:0/4189997590 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55972ed93b80 con 0x55972fb5b800 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50f86fa640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55972f0e5400 0x55972f04bb80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50de6c6640 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f15c000 0x55972f04cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50de6c6640 1 -- 172.21.15.138:0/4189997590 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55972ed7bdc0 con 0x55972fb5b800 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50f96fc640 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f15c000 0x55972f04cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50f86fa640 1 -- 172.21.15.138:0/764587690 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55972f164240 con 0x55972f0e5400 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55972f0e5c00 0x55972f04d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:2 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55972f15c400 con 0x55972f0e5c00 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50f8efb640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55972f0e5c00 0x55972f04d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50f96fc640 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f15c000 0x55972f04cc00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55972fcd7590 tx=0x55972fcd45a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.340+0000 7f50f8efb640 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55972f0e5c00 0x55972f04d180 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50f8efb640 1 -- 172.21.15.138:0/764587690 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(2 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x55972f164480 con 0x55972f0e5c00 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f0e4000 msgr2=0x55972f04c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f0e4000 0x55972f04c680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55972ee75380 tx=0x55972fcd4d80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 msgr2=0x55972f04b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55972fc256b0 tx=0x55972ee7b9e0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 shutdown_connections 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f0e4000 0x55972f04c680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 --2- 172.21.15.138:0/354590495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b000 0x55972f04b080 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 >> 172.21.15.138:0/354590495 conn(0x55972fb5ac00 msgr2=0x55972ee73000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 shutdown_connections 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/354590495 wait complete. 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f15c000 msgr2=0x55972f04cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f15c000 0x55972f04cc00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55972fcd7590 tx=0x55972fcd45a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 msgr2=0x55972f04b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.341+0000 7f50fca19880 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x55972fc52ab0 tx=0x55972ee7bce0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 shutdown_connections 2024-04-27T05:23:20.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972f15c000 0x55972f04cc00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/4189997590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972fb5b800 0x55972f04b600 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 >> 172.21.15.138:0/4189997590 conn(0x55972fb5b400 msgr2=0x55972ee73800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 shutdown_connections 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/4189997590 wait complete. 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55972f0e5400 msgr2=0x55972f04bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55972f0e5400 0x55972f04bb80 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55972f0e5c00 msgr2=0x55972f04d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55972f0e5c00 0x55972f04d180 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972fb5bc00 msgr2=0x55972f04c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972fb5bc00 0x55972f04c100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55972fc52660 tx=0x55972ee7bd40 comp rx=0 tx=0).stop 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 msgr2=0x55972f04ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x55972fc24c00 tx=0x55972ee7ac60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 shutdown_connections 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55972f0e5c00 0x55972f04d180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55972f0e5400 0x55972f04bb80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55972fb5bc00 0x55972f04c100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 --2- 172.21.15.138:0/764587690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55972ed69400 0x55972f04ab00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 >> 172.21.15.138:0/764587690 conn(0x55972ed68c00 msgr2=0x55972ee71000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 shutdown_connections 2024-04-27T05:23:20.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.342+0000 7f50fca19880 1 -- 172.21.15.138:0/764587690 wait complete. 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "control_pool": "default.rgw.control", 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T05:23:20.361 INFO:teuthology.orchestra.run.smithi138.stdout: "log_pool": "default.rgw.log", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "system_key": { 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "", 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "" 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.362 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_pools": [ 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "default-placement", 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": true 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.363 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.364 INFO:teuthology.orchestra.run.smithi138.stdout: "realm_id": "", 2024-04-27T05:23:20.364 INFO:teuthology.orchestra.run.smithi138.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T05:23:20.364 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:20.364 DEBUG:tasks.util.rgw: json result: {'id': '2f19b8ff-2608-4d86-8f1c-430623205476', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T05:23:20.364 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-04-27T05:23:20.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-04-27T05:23:20.364 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-04-27T05:23:20.441 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.441 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c05142880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bf6159f400 0x55bf61880580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c05142880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bf61876600 con 0x55bf6159f000 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c05142880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bf615de280 con 0x55bf6159f400 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c01342640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x55bf6159f400 0x55bf61880580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c01b43640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x55bf6159f000 0x55bf616a6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38288/0 (socket says 172.21.15.138:38288) 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c01b43640 1 -- 172.21.15.138:0/2991396690 learned_addr learned my addr 172.21.15.138:0/2991396690 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.440+0000 7f8c01342640 1 -- 172.21.15.138:0/2991396690 >> v1:172.21.15.114:6789/0 conn(0x55bf6159f000 legacy=0x55bf616a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.441+0000 7f8c01342640 1 -- 172.21.15.138:0/2991396690 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf615a3380 con 0x55bf6159f400 2024-04-27T05:23:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.442+0000 7f8c01342640 1 --2- 172.21.15.138:0/2991396690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f400 0x55bf61880580 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55bf616aac00 tx=0x55bf616b0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a48ada949fa32a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.442+0000 7f8c00b41640 1 -- 172.21.15.138:0/2991396690 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7ce0 con 0x55bf6159f400 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.442+0000 7f8c00b41640 1 -- 172.21.15.138:0/2991396690 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf61877b00 con 0x55bf6159f400 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.442+0000 7f8c00b41640 1 -- 172.21.15.138:0/2991396690 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7e40 con 0x55bf6159f400 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 -- 172.21.15.138:0/2991396690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f400 msgr2=0x55bf61880580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 --2- 172.21.15.138:0/2991396690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f400 0x55bf61880580 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55bf616aac00 tx=0x55bf616b0de0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 -- 172.21.15.138:0/2991396690 shutdown_connections 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 --2- 172.21.15.138:0/2991396690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f400 0x55bf61880580 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 -- 172.21.15.138:0/2991396690 >> 172.21.15.138:0/2991396690 conn(0x55bf6159ec00 msgr2=0x55bf616a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 -- 172.21.15.138:0/2991396690 shutdown_connections 2024-04-27T05:23:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 -- 172.21.15.138:0/2991396690 wait complete. 2024-04-27T05:23:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.443+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c05142880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c05142880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf615de780 con 0x55bf6159f000 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35480/0 (socket says 172.21.15.138:35480) 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c01b43640 1 -- 172.21.15.138:0/1730520731 learned_addr learned my addr 172.21.15.138:0/1730520731 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c01b43640 1 -- 172.21.15.138:0/1730520731 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf615a2ea0 con 0x55bf6159f000 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c01b43640 1 --2- 172.21.15.138:0/1730520731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55bf6245acc0 tx=0x55bf616b0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e05167ca8ae8b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c00340640 1 -- 172.21.15.138:0/1730520731 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b74a0 con 0x55bf6159f000 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c00340640 1 -- 172.21.15.138:0/1730520731 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf61877e00 con 0x55bf6159f000 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.444+0000 7f8c00340640 1 -- 172.21.15.138:0/1730520731 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7ce0 con 0x55bf6159f000 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- 172.21.15.138:0/1730520731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 msgr2=0x55bf61880580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 --2- 172.21.15.138:0/1730520731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55bf6245acc0 tx=0x55bf616b0cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- 172.21.15.138:0/1730520731 shutdown_connections 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 --2- 172.21.15.138:0/1730520731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- 172.21.15.138:0/1730520731 >> 172.21.15.138:0/1730520731 conn(0x55bf6159ec00 msgr2=0x55bf616a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- 172.21.15.138:0/1730520731 shutdown_connections 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- 172.21.15.138:0/1730520731 wait complete. 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c05142880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf615de8c0 con 0x55bf6159f000 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35492/0 (socket says 172.21.15.138:35492) 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.445+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 learned_addr learned my addr 172.21.15.138:0/2674956021 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf615a36c0 con 0x55bf6159f000 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8c01b43640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55bf6245aea0 tx=0x55bf616b0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8bfeb3d640 1 -- 172.21.15.138:0/2674956021 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7e40 con 0x55bf6159f000 2024-04-27T05:23:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8bfeb3d640 1 -- 172.21.15.138:0/2674956021 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62470000 con 0x55bf6159f000 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf615a31e0 con 0x55bf6159f000 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8bfeb3d640 1 -- 172.21.15.138:0/2674956021 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b74a0 con 0x55bf6159f000 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.446+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf61610ea0 con 0x55bf6159f000 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c05142880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8bfeb3d640 1 -- 172.21.15.138:0/2674956021 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bf615c9600 con 0x55bf6159f000 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c05142880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf615dec80 con 0x55bf62397800 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8bfeb3d640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf624f4000 0x55bf61881080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c02344640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8bfeb3d640 1 -- 172.21.15.138:0/2674956021 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bf615b1a40 con 0x55bf6159f000 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c02344640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35500/0 (socket says 172.21.15.138:35500) 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c02344640 1 -- 172.21.15.138:0/104103986 learned_addr learned my addr 172.21.15.138:0/104103986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c01b43640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf624f4000 0x55bf61881080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c02344640 1 -- 172.21.15.138:0/104103986 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf61610d00 con 0x55bf62397800 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c01b43640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf624f4000 0x55bf61881080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55bf624a4450 tx=0x55bf616b14a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.447+0000 7f8c02344640 1 --2- 172.21.15.138:0/104103986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x55bf6245bec0 tx=0x55bf616b1c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5ac9e61b906574 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8bfc338640 1 -- 172.21.15.138:0/104103986 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7e40 con 0x55bf62397800 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8bfc338640 1 -- 172.21.15.138:0/104103986 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62470300 con 0x55bf62397800 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8bfc338640 1 -- 172.21.15.138:0/104103986 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b74a0 con 0x55bf62397800 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- 172.21.15.138:0/104103986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 msgr2=0x55bf61880b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 --2- 172.21.15.138:0/104103986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x55bf6245bec0 tx=0x55bf616b1c80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- 172.21.15.138:0/104103986 shutdown_connections 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 --2- 172.21.15.138:0/104103986 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- 172.21.15.138:0/104103986 >> 172.21.15.138:0/104103986 conn(0x55bf6159f400 msgr2=0x55bf616a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- 172.21.15.138:0/104103986 shutdown_connections 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- 172.21.15.138:0/104103986 wait complete. 2024-04-27T05:23:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.448+0000 7f8c05142880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf615c2280 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c02344640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c02344640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35510/0 (socket says 172.21.15.138:35510) 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c02344640 1 -- 172.21.15.138:0/2850277760 learned_addr learned my addr 172.21.15.138:0/2850277760 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c02344640 1 -- 172.21.15.138:0/2850277760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf61610b60 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c02344640 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55bf6245b950 tx=0x55bf616b11a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8bfab35640 1 -- 172.21.15.138:0/2850277760 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7e40 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8bfab35640 1 -- 172.21.15.138:0/2850277760 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62470480 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf616109c0 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8bfab35640 1 -- 172.21.15.138:0/2850277760 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b74a0 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.449+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf61610820 con 0x55bf62397800 2024-04-27T05:23:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8bfab35640 1 -- 172.21.15.138:0/2850277760 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bf615c98c0 con 0x55bf62397800 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8bfab35640 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf61922000 0x55bf61881600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8bfab35640 1 -- 172.21.15.138:0/2850277760 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bf615b1c00 con 0x55bf62397800 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c05142880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c05142880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c01342640 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf61922000 0x55bf61881600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c05142880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c05142880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf615c3180 con 0x55bf61922400 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c01342640 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf61922000 0x55bf61881600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55bf616aac00 tx=0x55bf624fcd80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35518/0 (socket says 172.21.15.138:35518) 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.450+0000 7f8c01b43640 1 -- 172.21.15.138:0/4034003034 learned_addr learned my addr 172.21.15.138:0/4034003034 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c01b43640 1 -- 172.21.15.138:0/4034003034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf61610680 con 0x55bf61922400 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c01b43640 1 --2- 172.21.15.138:0/4034003034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55bf61904690 tx=0x55bf616b1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec87793032305ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8be8b11640 1 -- 172.21.15.138:0/4034003034 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7ce0 con 0x55bf61922400 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8be8b11640 1 -- 172.21.15.138:0/4034003034 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62470900 con 0x55bf61922400 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8be8b11640 1 -- 172.21.15.138:0/4034003034 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7e40 con 0x55bf61922400 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c05142880 1 -- 172.21.15.138:0/4034003034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 msgr2=0x55bf61881b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c05142880 1 --2- 172.21.15.138:0/4034003034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55bf61904690 tx=0x55bf616b1e00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c05142880 1 -- 172.21.15.138:0/4034003034 shutdown_connections 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c05142880 1 --2- 172.21.15.138:0/4034003034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c05142880 1 -- 172.21.15.138:0/4034003034 >> 172.21.15.138:0/4034003034 conn(0x55bf62397c00 msgr2=0x55bf616a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.451+0000 7f8c05142880 1 -- 172.21.15.138:0/4034003034 shutdown_connections 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c05142880 1 -- 172.21.15.138:0/4034003034 wait complete. 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c05142880 1 Processor -- start 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c05142880 1 -- start start 2024-04-27T05:23:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c05142880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c05142880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf615c2b40 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c01b43640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35530/0 (socket says 172.21.15.138:35530) 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c01b43640 1 -- 172.21.15.138:0/3715363474 learned_addr learned my addr 172.21.15.138:0/3715363474 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c01b43640 1 -- 172.21.15.138:0/3715363474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf616104e0 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.452+0000 7f8c01b43640 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55bf62514780 tx=0x55bf616b1440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be730e640 1 -- 172.21.15.138:0/3715363474 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b74a0 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be730e640 1 -- 172.21.15.138:0/3715363474 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62470000 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be730e640 1 -- 172.21.15.138:0/3715363474 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf615b7ce0 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf615a2ea0 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf615a36c0 con 0x55bf61922400 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be530a640 20 reqs_thread_entry: start 2024-04-27T05:23:20.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bf61922c00 0x55bf61882100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf61923000 con 0x55bf61922c00 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c01342640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bf61922c00 0x55bf61882100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be730e640 1 -- 172.21.15.138:0/3715363474 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x55bf6196a000 con 0x55bf61922400 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c01342640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bf61922c00 0x55bf61882100 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be730e640 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf6199c000 0x55bf61882c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8be730e640 1 -- 172.21.15.138:0/3715363474 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x55bf615b1dc0 con 0x55bf61922400 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c02344640 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf6199c000 0x55bf61882c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.453+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf61923400 con 0x55bf61922c00 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c02344640 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf6199c000 0x55bf61882c00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55bf62515680 tx=0x55bf624fc5a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bf6199d000 0x55bf61882680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c01b43640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bf6199d000 0x55bf61882680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c01b43640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bf6199d000 0x55bf61882680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.454+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d800 con 0x55bf61922c00 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:NOTICE: set zone's realm_id= 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199dc00 con 0x55bf6199d000 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(5 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c01b43640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bf6199dc00 0x55bf61883180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199dc00 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.455+0000 7f8c01342640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bf6199dc00 0x55bf61883180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.456+0000 7f8c01342640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bf6199dc00 0x55bf61883180 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.456+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf6199dc00 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.456+0000 7f8c01342640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bf6199d800 0x55bf61883700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.456+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.456+0000 7f8c02344640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bf6199d800 0x55bf61883700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.456+0000 7f8c02344640 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bf6199d800 0x55bf61883700 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.457+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199dc00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.458+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf6199dc00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.458+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.458+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(13 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.458+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.458+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.458+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199dc00 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf6199dc00 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.459+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(22 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.460+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199dc00 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf6199dc00 2024-04-27T05:23:20.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.461+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199dc00 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf6199dc00 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.462+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d000 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199d800 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193b200 con 0x55bf6199d800 2024-04-27T05:23:20.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf61922c00 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(35 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf6199d400 con 0x55bf6199dc00 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(36 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf6193a900 con 0x55bf6199dc00 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.463+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf619b8000 con 0x55bf61922c00 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.464+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.464+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:38 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf619b8400 con 0x55bf6199d000 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.464+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(38 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.464+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:39 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [create,call version.set in=58b,writefull 0~1036 in=1036b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf619b8800 con 0x55bf6199d000 2024-04-27T05:23:20.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.465+0000 7f8c01b43640 1 -- 172.21.15.138:0/2674956021 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(39 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [create,call,writefull 0~1036] v31'3 uv3 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf6193afc0 con 0x55bf6199d000 2024-04-27T05:23:20.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.465+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf619b8c00 con 0x55bf61922c00 2024-04-27T05:23:20.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.466+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v31'3 uv3 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.466+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bf619b9000 con 0x55bf61922c00 2024-04-27T05:23:20.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.467+0000 7f8c01342640 1 -- 172.21.15.138:0/2674956021 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v31'1 uv1 ondisk = 0) v8 ==== 199+0+0 (crc 0 0 0) 0x55bf6193a900 con 0x55bf61922c00 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf61922000 msgr2=0x55bf61881600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf61922000 0x55bf61881600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55bf616aac00 tx=0x55bf624fcd80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 msgr2=0x55bf61880b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55bf6245b950 tx=0x55bf616b11a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 shutdown_connections 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf61922000 0x55bf61881600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/2850277760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf62397800 0x55bf61880b00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 >> 172.21.15.138:0/2850277760 conn(0x55bf6159f400 msgr2=0x55bf616a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 shutdown_connections 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/2850277760 wait complete. 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf6199c000 msgr2=0x55bf61882c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf6199c000 0x55bf61882c00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55bf62515680 tx=0x55bf624fc5a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 msgr2=0x55bf61881b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55bf62514780 tx=0x55bf616b1440 comp rx=0 tx=0).stop 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 shutdown_connections 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf6199c000 0x55bf61882c00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 --2- 172.21.15.138:0/3715363474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf61922400 0x55bf61881b80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 >> 172.21.15.138:0/3715363474 conn(0x55bf62397c00 msgr2=0x55bf616a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 shutdown_connections 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.468+0000 7f8c05142880 1 -- 172.21.15.138:0/3715363474 wait complete. 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bf6199d800 msgr2=0x55bf61883700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x55bf6199d800 0x55bf61883700 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bf61922c00 msgr2=0x55bf61882100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x55bf61922c00 0x55bf61882100 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bf6199dc00 msgr2=0x55bf61883180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x55bf6199dc00 0x55bf61883180 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bf6199d000 msgr2=0x55bf61882680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x55bf6199d000 0x55bf61882680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf624f4000 msgr2=0x55bf61881080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x55bf624f4000 0x55bf61881080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55bf624a4450 tx=0x55bf616b14a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 msgr2=0x55bf61880580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c02344640 1 -- 172.21.15.138:0/2674956021 reap_dead start 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55bf6245aea0 tx=0x55bf616b0c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 shutdown_connections 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 --2- 172.21.15.138:0/2674956021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bf6159f000 0x55bf61880580 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 >> 172.21.15.138:0/2674956021 conn(0x55bf6159ec00 msgr2=0x55bf616a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 shutdown_connections 2024-04-27T05:23:20.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.469+0000 7f8c05142880 1 -- 172.21.15.138:0/2674956021 wait complete. 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "control_pool": "default.rgw.control", 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T05:23:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: "log_pool": "default.rgw.log", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "system_key": { 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "", 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "" 2024-04-27T05:23:20.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_pools": [ 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "default-placement", 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": true 2024-04-27T05:23:20.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "new-placement", 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": ".rgw.buckets.index.2", 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": ".rgw.buckets.2" 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": false 2024-04-27T05:23:20.491 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.492 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.492 INFO:teuthology.orchestra.run.smithi138.stdout: "realm_id": "", 2024-04-27T05:23:20.492 INFO:teuthology.orchestra.run.smithi138.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T05:23:20.492 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:20.492 DEBUG:tasks.util.rgw: json result: {'id': '2f19b8ff-2608-4d86-8f1c-430623205476', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T05:23:20.492 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-04-27T05:23:20.492 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-04-27T05:23:20.493 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-04-27T05:23:20.568 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.569 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f367b538880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5570c6821400 0x5570c6b02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f367b538880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5570c6af8600 con 0x5570c6821000 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f367b538880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5570c6860280 con 0x5570c6821400 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f3677713640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x5570c6821400 0x5570c6b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f3677f14640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x5570c6821000 0x5570c6928800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38300/0 (socket says 172.21.15.138:38300) 2024-04-27T05:23:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f3677f14640 1 -- 172.21.15.138:0/1805291089 learned_addr learned my addr 172.21.15.138:0/1805291089 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:23:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f3677713640 1 -- 172.21.15.138:0/1805291089 >> v1:172.21.15.114:6789/0 conn(0x5570c6821000 legacy=0x5570c6928800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f3677713640 1 -- 172.21.15.138:0/1805291089 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c6825380 con 0x5570c6821400 2024-04-27T05:23:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.567+0000 7f3676f12640 1 -- 172.21.15.138:0/1805291089 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911310001 0 0) 0x5570c68249c0 con 0x5570c6821000 2024-04-27T05:23:20.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.569+0000 7f3677713640 1 --2- 172.21.15.138:0/1805291089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821400 0x5570c6b02580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x5570c692cc00 tx=0x5570c6932de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749b50b92f3601fa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.569+0000 7f3676f12640 1 -- 172.21.15.138:0/1805291089 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839ce0 con 0x5570c6821400 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.569+0000 7f3676f12640 1 -- 172.21.15.138:0/1805291089 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c6af9b00 con 0x5570c6821400 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.569+0000 7f3676f12640 1 -- 172.21.15.138:0/1805291089 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839e40 con 0x5570c6821400 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 -- 172.21.15.138:0/1805291089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821400 msgr2=0x5570c6b02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 --2- 172.21.15.138:0/1805291089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821400 0x5570c6b02580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x5570c692cc00 tx=0x5570c6932de0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 -- 172.21.15.138:0/1805291089 shutdown_connections 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 --2- 172.21.15.138:0/1805291089 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821400 0x5570c6b02580 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 -- 172.21.15.138:0/1805291089 >> 172.21.15.138:0/1805291089 conn(0x5570c6820c00 msgr2=0x5570c6928000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 -- 172.21.15.138:0/1805291089 shutdown_connections 2024-04-27T05:23:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.570+0000 7f367b538880 1 -- 172.21.15.138:0/1805291089 wait complete. 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f367b538880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f367b538880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c6860780 con 0x5570c6821000 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35546/0 (socket says 172.21.15.138:35546) 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3677f14640 1 -- 172.21.15.138:0/2441255590 learned_addr learned my addr 172.21.15.138:0/2441255590 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3677f14640 1 -- 172.21.15.138:0/2441255590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c6824ea0 con 0x5570c6821000 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3677f14640 1 --2- 172.21.15.138:0/2441255590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5570c76dcc90 tx=0x5570c6932cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928b5f09adedb71b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3676711640 1 -- 172.21.15.138:0/2441255590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c68394a0 con 0x5570c6821000 2024-04-27T05:23:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3676711640 1 -- 172.21.15.138:0/2441255590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c6af9e00 con 0x5570c6821000 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.571+0000 7f3676711640 1 -- 172.21.15.138:0/2441255590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839ce0 con 0x5570c6821000 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- 172.21.15.138:0/2441255590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 msgr2=0x5570c6b02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 --2- 172.21.15.138:0/2441255590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5570c76dcc90 tx=0x5570c6932cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- 172.21.15.138:0/2441255590 shutdown_connections 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 --2- 172.21.15.138:0/2441255590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- 172.21.15.138:0/2441255590 >> 172.21.15.138:0/2441255590 conn(0x5570c6820c00 msgr2=0x5570c6929000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- 172.21.15.138:0/2441255590 shutdown_connections 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- 172.21.15.138:0/2441255590 wait complete. 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f367b538880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c68608c0 con 0x5570c6821000 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35550/0 (socket says 172.21.15.138:35550) 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.572+0000 7f3677f14640 1 -- 172.21.15.138:0/2332860274 learned_addr learned my addr 172.21.15.138:0/2332860274 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f3677f14640 1 -- 172.21.15.138:0/2332860274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c68256c0 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f3677f14640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x5570c76dd350 tx=0x5570c6932c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f3674f0e640 1 -- 172.21.15.138:0/2332860274 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839e40 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f3674f0e640 1 -- 172.21.15.138:0/2332860274 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c76f2000 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5570c68251e0 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f3674f0e640 1 -- 172.21.15.138:0/2332860274 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c68394a0 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.573+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570c6892ea0 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3674f0e640 1 -- 172.21.15.138:0/2332860274 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5570c684b600 con 0x5570c6821000 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f367b538880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f367b538880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c6860c80 con 0x5570c7619800 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3674f0e640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c7770000 0x5570c6b03080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3674f0e640 1 -- 172.21.15.138:0/2332860274 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5570c6833a40 con 0x5570c6821000 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3677f14640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c7770000 0x5570c6b03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3678715640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3678715640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35560/0 (socket says 172.21.15.138:35560) 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3678715640 1 -- 172.21.15.138:0/3040214672 learned_addr learned my addr 172.21.15.138:0/3040214672 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3678715640 1 -- 172.21.15.138:0/3040214672 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c6892d00 con 0x5570c7619800 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3677f14640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c7770000 0x5570c6b03080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5570c7724420 tx=0x5570c69334a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3678715640 1 --2- 172.21.15.138:0/3040214672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x5570c76dd560 tx=0x5570c6933f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a2ebc795bff775 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.574+0000 7f3672709640 1 -- 172.21.15.138:0/3040214672 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839e40 con 0x5570c7619800 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f3672709640 1 -- 172.21.15.138:0/3040214672 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c76f2300 con 0x5570c7619800 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f3672709640 1 -- 172.21.15.138:0/3040214672 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c68394a0 con 0x5570c7619800 2024-04-27T05:23:20.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- 172.21.15.138:0/3040214672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 msgr2=0x5570c6b02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 --2- 172.21.15.138:0/3040214672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x5570c76dd560 tx=0x5570c6933f80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- 172.21.15.138:0/3040214672 shutdown_connections 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 --2- 172.21.15.138:0/3040214672 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- 172.21.15.138:0/3040214672 >> 172.21.15.138:0/3040214672 conn(0x5570c6821400 msgr2=0x5570c692b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- 172.21.15.138:0/3040214672 shutdown_connections 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- 172.21.15.138:0/3040214672 wait complete. 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f367b538880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c6844280 con 0x5570c7619800 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.575+0000 7f3678715640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3678715640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35568/0 (socket says 172.21.15.138:35568) 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3678715640 1 -- 172.21.15.138:0/3465419720 learned_addr learned my addr 172.21.15.138:0/3465419720 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3678715640 1 -- 172.21.15.138:0/3465419720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c6892b60 con 0x5570c7619800 2024-04-27T05:23:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3678715640 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5570c76dd8f0 tx=0x5570c6933bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3670f06640 1 -- 172.21.15.138:0/3465419720 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839e40 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3670f06640 1 -- 172.21.15.138:0/3465419720 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c76f2480 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f3670f06640 1 -- 172.21.15.138:0/3465419720 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c68394a0 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5570c68929c0 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.576+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570c6892820 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f3670f06640 1 -- 172.21.15.138:0/3465419720 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5570c684b8c0 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f3670f06640 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6bb2000 0x5570c6b03600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f367b538880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f367b538880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f3670f06640 1 -- 172.21.15.138:0/3465419720 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5570c6833c00 con 0x5570c7619800 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f3677713640 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6bb2000 0x5570c6b03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f367b538880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f367b538880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c6845180 con 0x5570c6bb2400 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.577+0000 7f3677713640 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6bb2000 0x5570c6b03600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5570c692cc00 tx=0x5570c779ee40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35580/0 (socket says 172.21.15.138:35580) 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f3677f14640 1 -- 172.21.15.138:0/3664776253 learned_addr learned my addr 172.21.15.138:0/3664776253 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f3677f14640 1 -- 172.21.15.138:0/3664776253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c6892680 con 0x5570c6bb2400 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f3677f14640 1 --2- 172.21.15.138:0/3664776253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x5570c77245a0 tx=0x5570c6933e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1465de5206369047 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f365eee2640 1 -- 172.21.15.138:0/3664776253 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839ce0 con 0x5570c6bb2400 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f365eee2640 1 -- 172.21.15.138:0/3664776253 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c76f2900 con 0x5570c6bb2400 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f365eee2640 1 -- 172.21.15.138:0/3664776253 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839e40 con 0x5570c6bb2400 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f367b538880 1 -- 172.21.15.138:0/3664776253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 msgr2=0x5570c6b03b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f367b538880 1 --2- 172.21.15.138:0/3664776253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x5570c77245a0 tx=0x5570c6933e60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f367b538880 1 -- 172.21.15.138:0/3664776253 shutdown_connections 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f367b538880 1 --2- 172.21.15.138:0/3664776253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f367b538880 1 -- 172.21.15.138:0/3664776253 >> 172.21.15.138:0/3664776253 conn(0x5570c7619c00 msgr2=0x5570c692b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.578+0000 7f367b538880 1 -- 172.21.15.138:0/3664776253 shutdown_connections 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f367b538880 1 -- 172.21.15.138:0/3664776253 wait complete. 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f367b538880 1 Processor -- start 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f367b538880 1 -- start start 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f367b538880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f367b538880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570c6844b40 con 0x5570c6bb2400 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f3677f14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35592/0 (socket says 172.21.15.138:35592) 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f3677f14640 1 -- 172.21.15.138:0/4004452243 learned_addr learned my addr 172.21.15.138:0/4004452243 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.579+0000 7f3677f14640 1 -- 172.21.15.138:0/4004452243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570c6824ea0 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f3677f14640 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x5570c77a0f90 tx=0x5570c6933440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f365d6df640 1 -- 172.21.15.138:0/4004452243 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c68394a0 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f365d6df640 1 -- 172.21.15.138:0/4004452243 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5570c76f2000 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5570c68256c0 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f365d6df640 1 -- 172.21.15.138:0/4004452243 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5570c6839ce0 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570c68251e0 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f365b6db640 20 reqs_thread_entry: start 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5570c6bb2c00 0x5570c6b04100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5570c6bb3000 con 0x5570c6bb2c00 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f3677713640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5570c6bb2c00 0x5570c6b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f365d6df640 1 -- 172.21.15.138:0/4004452243 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5570c6bea000 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.580+0000 7f3677713640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5570c6bb2c00 0x5570c6b04100 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f365d6df640 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6c1a000 0x5570c6b04c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f365d6df640 1 -- 172.21.15.138:0/4004452243 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x5570c6833dc0 con 0x5570c6bb2400 2024-04-27T05:23:20.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f3678715640 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6c1a000 0x5570c6b04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f3677713640 1 -- 172.21.15.138:0/2332860274 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5570c6bc2b40 con 0x5570c6bb2c00 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5570c6bb3400 con 0x5570c6bb2c00 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f3677713640 1 -- 172.21.15.138:0/2332860274 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5570c6bc2b40 con 0x5570c6bb2c00 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f3678715640 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6c1a000 0x5570c6b04c00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5570c77a1680 tx=0x5570c779fc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5570c6c1b000 0x5570c6b04680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5570c6c1b400 con 0x5570c6c1b000 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f3677f14640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5570c6c1b000 0x5570c6b04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.581+0000 7f3677f14640 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5570c6c1b000 0x5570c6b04680 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f3677f14640 1 -- 172.21.15.138:0/2332860274 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x5570c6bc3200 con 0x5570c6c1b000 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6bb2000 msgr2=0x5570c6b03600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6bb2000 0x5570c6b03600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5570c692cc00 tx=0x5570c779ee40 comp rx=0 tx=0).stop 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 msgr2=0x5570c6b02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x5570c76dd8f0 tx=0x5570c6933bc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 shutdown_connections 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6bb2000 0x5570c6b03600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 --2- 172.21.15.138:0/3465419720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c7619800 0x5570c6b02b00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 >> 172.21.15.138:0/3465419720 conn(0x5570c6821400 msgr2=0x5570c692b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 shutdown_connections 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.582+0000 7f367b538880 1 -- 172.21.15.138:0/3465419720 wait complete. 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6c1a000 msgr2=0x5570c6b04c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6c1a000 0x5570c6b04c00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5570c77a1680 tx=0x5570c779fc20 comp rx=0 tx=0).stop 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 msgr2=0x5570c6b03b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x5570c77a0f90 tx=0x5570c6933440 comp rx=0 tx=0).stop 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 shutdown_connections 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c6c1a000 0x5570c6b04c00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/4004452243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6bb2400 0x5570c6b03b80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 >> 172.21.15.138:0/4004452243 conn(0x5570c7619c00 msgr2=0x5570c692b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 shutdown_connections 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/4004452243 wait complete. 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5570c6bb2c00 msgr2=0x5570c6b04100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5570c6bb2c00 0x5570c6b04100 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5570c6c1b000 msgr2=0x5570c6b04680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5570c6c1b000 0x5570c6b04680 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c7770000 msgr2=0x5570c6b03080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c7770000 0x5570c6b03080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5570c7724420 tx=0x5570c69334a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 msgr2=0x5570c6b02580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x5570c76dd350 tx=0x5570c6932c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 shutdown_connections 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x5570c6c1b000 0x5570c6b04680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x5570c6bb2c00 0x5570c6b04100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5570c7770000 0x5570c6b03080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 --2- 172.21.15.138:0/2332860274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5570c6821000 0x5570c6b02580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 >> 172.21.15.138:0/2332860274 conn(0x5570c6820c00 msgr2=0x5570c6929000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 shutdown_connections 2024-04-27T05:23:20.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.583+0000 7f367b538880 1 -- 172.21.15.138:0/2332860274 wait complete. 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "control_pool": "default.rgw.control", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "log_pool": "default.rgw.log", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T05:23:20.601 INFO:teuthology.orchestra.run.smithi138.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "system_key": { 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "" 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_pools": [ 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "default-placement", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T05:23:20.602 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": true 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "new-placement", 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": ".rgw.buckets.index.2", 2024-04-27T05:23:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": ".rgw.buckets.2" 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": false 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "realm_id": "", 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T05:23:20.604 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:20.605 DEBUG:tasks.util.rgw: json result: {'id': '2f19b8ff-2608-4d86-8f1c-430623205476', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T05:23:20.605 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T05:23:20.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T05:23:20.605 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-27T05:23:20.685 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.685 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.683+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f579d41d880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x561356c95400 0x561356f76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f579d41d880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561356f6c600 con 0x561356c95000 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f579d41d880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x561356cd4280 con 0x561356c95400 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f5799101640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x561356c95400 0x561356f76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f5799902640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x561356c95000 0x561356d9c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38304/0 (socket says 172.21.15.138:38304) 2024-04-27T05:23:20.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f5799101640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x561356c95400 0x561356f76580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35602/0 (socket says 172.21.15.138:35602) 2024-04-27T05:23:20.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f5799902640 1 -- 172.21.15.138:0/3366522664 learned_addr learned my addr 172.21.15.138:0/3366522664 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-27T05:23:20.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f5799101640 1 -- 172.21.15.138:0/3366522664 >> v1:172.21.15.114:6789/0 conn(0x561356c95000 legacy=0x561356d9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.684+0000 7f5799101640 1 -- 172.21.15.138:0/3366522664 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356c99380 con 0x561356c95400 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.686+0000 7f5799101640 1 --2- 172.21.15.138:0/3366522664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95400 0x561356f76580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x561356da0c00 tx=0x561356da6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e75b04a2775bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.686+0000 7f5798900640 1 -- 172.21.15.138:0/3366522664 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cadce0 con 0x561356c95400 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.686+0000 7f5798900640 1 -- 172.21.15.138:0/3366522664 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561356f6db00 con 0x561356c95400 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.686+0000 7f5798900640 1 -- 172.21.15.138:0/3366522664 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cade40 con 0x561356c95400 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 -- 172.21.15.138:0/3366522664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95400 msgr2=0x561356f76580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 --2- 172.21.15.138:0/3366522664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95400 0x561356f76580 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x561356da0c00 tx=0x561356da6de0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 -- 172.21.15.138:0/3366522664 shutdown_connections 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 --2- 172.21.15.138:0/3366522664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95400 0x561356f76580 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 -- 172.21.15.138:0/3366522664 >> 172.21.15.138:0/3366522664 conn(0x561356c94c00 msgr2=0x561356d9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 -- 172.21.15.138:0/3366522664 shutdown_connections 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 -- 172.21.15.138:0/3366522664 wait complete. 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.687+0000 7f579d41d880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f579d41d880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561356cd4780 con 0x561356c95000 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35612/0 (socket says 172.21.15.138:35612) 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f5799902640 1 -- 172.21.15.138:0/1401336947 learned_addr learned my addr 172.21.15.138:0/1401336947 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f5799902640 1 -- 172.21.15.138:0/1401336947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356c98ea0 con 0x561356c95000 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f5799902640 1 --2- 172.21.15.138:0/1401336947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x561357b50cc0 tx=0x561356da6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3376e44eb73323ed server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f57980ff640 1 -- 172.21.15.138:0/1401336947 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cad4a0 con 0x561356c95000 2024-04-27T05:23:20.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f57980ff640 1 -- 172.21.15.138:0/1401336947 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561356f6de00 con 0x561356c95000 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.688+0000 7f57980ff640 1 -- 172.21.15.138:0/1401336947 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cadce0 con 0x561356c95000 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- 172.21.15.138:0/1401336947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 msgr2=0x561356f76580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 --2- 172.21.15.138:0/1401336947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x561357b50cc0 tx=0x561356da6cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- 172.21.15.138:0/1401336947 shutdown_connections 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 --2- 172.21.15.138:0/1401336947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- 172.21.15.138:0/1401336947 >> 172.21.15.138:0/1401336947 conn(0x561356c94c00 msgr2=0x561356d9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- 172.21.15.138:0/1401336947 shutdown_connections 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- 172.21.15.138:0/1401336947 wait complete. 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f579d41d880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561356cd48c0 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35624/0 (socket says 172.21.15.138:35624) 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.689+0000 7f5799902640 1 -- 172.21.15.138:0/726905365 learned_addr learned my addr 172.21.15.138:0/726905365 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f5799902640 1 -- 172.21.15.138:0/726905365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356c996c0 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f5799902640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x561357b50ea0 tx=0x561356da6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f57968fc640 1 -- 172.21.15.138:0/726905365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cade40 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f57968fc640 1 -- 172.21.15.138:0/726905365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561357b66000 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f57968fc640 1 -- 172.21.15.138:0/726905365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cad4a0 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561356c991e0 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.690+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561356d06ea0 con 0x561356c95000 2024-04-27T05:23:20.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f57968fc640 1 -- 172.21.15.138:0/726905365 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561356cbf600 con 0x561356c95000 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579d41d880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579d41d880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561356cd4c80 con 0x561357a8d800 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f57968fc640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357bec000 0x561356f77080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f57968fc640 1 -- 172.21.15.138:0/726905365 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x561356ca7a40 con 0x561356c95000 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579a103640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579a103640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35634/0 (socket says 172.21.15.138:35634) 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579a103640 1 -- 172.21.15.138:0/1558375917 learned_addr learned my addr 172.21.15.138:0/1558375917 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f5799902640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357bec000 0x561356f77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579a103640 1 -- 172.21.15.138:0/1558375917 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356d06d00 con 0x561357a8d800 2024-04-27T05:23:20.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f5799902640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357bec000 0x561356f77080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x561357b9a450 tx=0x561356da74a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f579a103640 1 --2- 172.21.15.138:0/1558375917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x561357b51f50 tx=0x561356da7c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03d269edc739afe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f57940f7640 1 -- 172.21.15.138:0/1558375917 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cade40 con 0x561357a8d800 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.691+0000 7f57940f7640 1 -- 172.21.15.138:0/1558375917 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561357b66300 con 0x561357a8d800 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f57940f7640 1 -- 172.21.15.138:0/1558375917 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cad4a0 con 0x561357a8d800 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- 172.21.15.138:0/1558375917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 msgr2=0x561356f76b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 --2- 172.21.15.138:0/1558375917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x561357b51f50 tx=0x561356da7c80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- 172.21.15.138:0/1558375917 shutdown_connections 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 --2- 172.21.15.138:0/1558375917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- 172.21.15.138:0/1558375917 >> 172.21.15.138:0/1558375917 conn(0x561356c95400 msgr2=0x561356d9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- 172.21.15.138:0/1558375917 shutdown_connections 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- 172.21.15.138:0/1558375917 wait complete. 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579d41d880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561356cb8280 con 0x561357a8d800 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579a103640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.692+0000 7f579a103640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35642/0 (socket says 172.21.15.138:35642) 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f579a103640 1 -- 172.21.15.138:0/2764436059 learned_addr learned my addr 172.21.15.138:0/2764436059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f579a103640 1 -- 172.21.15.138:0/2764436059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356d06b60 con 0x561357a8d800 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f579a103640 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x561357a99470 tx=0x561356da71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f57928f4640 1 -- 172.21.15.138:0/2764436059 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cade40 con 0x561357a8d800 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f57928f4640 1 -- 172.21.15.138:0/2764436059 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561357b66480 con 0x561357a8d800 2024-04-27T05:23:20.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f57928f4640 1 -- 172.21.15.138:0/2764436059 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cad4a0 con 0x561357a8d800 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561356d069c0 con 0x561357a8d800 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.693+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561356d06820 con 0x561357a8d800 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f57928f4640 1 -- 172.21.15.138:0/2764436059 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x561356cbf8c0 con 0x561357a8d800 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f57928f4640 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357018000 0x561356f77600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f57928f4640 1 -- 172.21.15.138:0/2764436059 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x561356ca7c00 con 0x561357a8d800 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f579d41d880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f579d41d880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f5799101640 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357018000 0x561356f77600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f579d41d880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f5799101640 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357018000 0x561356f77600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x561356da0c00 tx=0x561357bf6d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.694+0000 7f579d41d880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561356cb9180 con 0x561357018400 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35644/0 (socket says 172.21.15.138:35644) 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f5799902640 1 -- 172.21.15.138:0/353273691 learned_addr learned my addr 172.21.15.138:0/353273691 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f5799902640 1 -- 172.21.15.138:0/353273691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356d06680 con 0x561357018400 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f5799902640 1 --2- 172.21.15.138:0/353273691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x561356ffa690 tx=0x561356da7e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce65ca7fc018254 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f57808d0640 1 -- 172.21.15.138:0/353273691 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cadce0 con 0x561357018400 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f57808d0640 1 -- 172.21.15.138:0/353273691 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561357b66900 con 0x561357018400 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.695+0000 7f57808d0640 1 -- 172.21.15.138:0/353273691 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cade40 con 0x561357018400 2024-04-27T05:23:20.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- 172.21.15.138:0/353273691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 msgr2=0x561356f77b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 --2- 172.21.15.138:0/353273691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x561356ffa690 tx=0x561356da7e00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- 172.21.15.138:0/353273691 shutdown_connections 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 --2- 172.21.15.138:0/353273691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- 172.21.15.138:0/353273691 >> 172.21.15.138:0/353273691 conn(0x561357a8dc00 msgr2=0x561356d9f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- 172.21.15.138:0/353273691 shutdown_connections 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- 172.21.15.138:0/353273691 wait complete. 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 Processor -- start 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- start start 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f579d41d880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x561356cb8b40 con 0x561357018400 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f5799902640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35656/0 (socket says 172.21.15.138:35656) 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.696+0000 7f5799902640 1 -- 172.21.15.138:0/2814568766 learned_addr learned my addr 172.21.15.138:0/2814568766 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f5799902640 1 -- 172.21.15.138:0/2814568766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561356d064e0 con 0x561357018400 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f5799902640 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x561357c0c780 tx=0x561356da7440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f577f0cd640 1 -- 172.21.15.138:0/2814568766 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cad4a0 con 0x561357018400 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f577f0cd640 1 -- 172.21.15.138:0/2814568766 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561357b66000 con 0x561357018400 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561356c98ea0 con 0x561357018400 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f577f0cd640 1 -- 172.21.15.138:0/2814568766 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561356cadce0 con 0x561357018400 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561356c996c0 con 0x561357018400 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f577d0c9640 20 reqs_thread_entry: start 2024-04-27T05:23:20.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561357018c00 0x561356f78100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561357019000 con 0x561357018c00 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f5799101640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561357018c00 0x561356f78100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.697+0000 7f577f0cd640 1 -- 172.21.15.138:0/2814568766 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x56135705e000 con 0x561357018400 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f5799101640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561357018c00 0x561356f78100 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f577f0cd640 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357092000 0x561356f78c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f577f0cd640 1 -- 172.21.15.138:0/2814568766 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x561356ca7dc0 con 0x561357018400 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f579a103640 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357092000 0x561356f78c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f5799101640 1 -- 172.21.15.138:0/726905365 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561357030480 con 0x561357018c00 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561357019400 0x561356f78680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:2 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561357092400 con 0x561357019400 2024-04-27T05:23:20.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f5799902640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561357019400 0x561356f78680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f579a103640 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357092000 0x561356f78c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x561357c0d680 tx=0x561357bf65a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.698+0000 7f5799902640 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561357019400 0x561356f78680 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.699+0000 7f5799902640 1 -- 172.21.15.138:0/726905365 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(2 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x5613570306c0 con 0x561357019400 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.699+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561357092800 con 0x561357019400 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f5799902640 1 -- 172.21.15.138:0/726905365 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(3 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [stat,call,call,writefull 0~874] v31'4 uv4 ondisk = 0) v8 ==== 316+0+0 (crc 0 0 0) 0x5613570306c0 con 0x561357019400 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357018000 msgr2=0x561356f77600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357018000 0x561356f77600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x561356da0c00 tx=0x561357bf6d80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 msgr2=0x561356f76b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x561357a99470 tx=0x561356da71a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 shutdown_connections 2024-04-27T05:23:20.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357018000 0x561356f77600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 --2- 172.21.15.138:0/2764436059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357a8d800 0x561356f76b00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.700+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 >> 172.21.15.138:0/2764436059 conn(0x561356c95400 msgr2=0x561356d9f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 shutdown_connections 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2764436059 wait complete. 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357092000 msgr2=0x561356f78c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357092000 0x561356f78c00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x561357c0d680 tx=0x561357bf65a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 msgr2=0x561356f77b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x561357c0c780 tx=0x561356da7440 comp rx=0 tx=0).stop 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 shutdown_connections 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357092000 0x561356f78c00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/2814568766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561357018400 0x561356f77b80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 >> 172.21.15.138:0/2814568766 conn(0x561357a8dc00 msgr2=0x561356d9f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 shutdown_connections 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/2814568766 wait complete. 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561357018c00 msgr2=0x561356f78100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561357018c00 0x561356f78100 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561357019400 msgr2=0x561356f78680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561357019400 0x561356f78680 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357bec000 msgr2=0x561356f77080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357bec000 0x561356f77080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x561357b9a450 tx=0x561356da74a0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 msgr2=0x561356f76580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x561357b50ea0 tx=0x561356da6c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 shutdown_connections 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x561357019400 0x561356f78680 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x561357018c00 0x561356f78100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x561357bec000 0x561356f77080 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 --2- 172.21.15.138:0/726905365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561356c95000 0x561356f76580 secure :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0x561357b50ea0 tx=0x561356da6c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 >> 172.21.15.138:0/726905365 conn(0x561356c94c00 msgr2=0x561356d9d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 shutdown_connections 2024-04-27T05:23:20.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.701+0000 7f579d41d880 1 -- 172.21.15.138:0/726905365 wait complete. 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "domain_root": "default.rgw.meta:root", 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "control_pool": "default.rgw.control", 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "gc_pool": "default.rgw.log:gc", 2024-04-27T05:23:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "lc_pool": "default.rgw.log:lc", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "log_pool": "default.rgw.log", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "roles_pool": "default.rgw.meta:roles", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "otp_pool": "default.rgw.otp", 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "system_key": { 2024-04-27T05:23:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "", 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "" 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_pools": [ 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "default-placement", 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "val": { 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "index_pool": "default.rgw.buckets.index", 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": { 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD": { 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "data_pool": "default.rgw.buckets.data" 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "index_type": 0, 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "inline_data": true 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "realm_id": "", 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "notif_pool": "default.rgw.log:notif" 2024-04-27T05:23:20.725 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:20.725 DEBUG:tasks.util.rgw: json result: {'id': '2f19b8ff-2608-4d86-8f1c-430623205476', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-04-27T05:23:20.725 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-04-27T05:23:20.726 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-04-27T05:23:20.726 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-04-27T05:23:20.766 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.766 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351f526880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351f526880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562804d72600 con 0x562804a9b400 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351f526880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x562804ada280 con 0x562804a9b000 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351bdc3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351bdc3640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35672/0 (socket says 172.21.15.138:35672) 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351bdc3640 1 -- 172.21.15.138:0/4173658470 learned_addr learned my addr 172.21.15.138:0/4173658470 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351b5c2640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x562804a9b400 0x562804ba2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38310/0 (socket says 172.21.15.138:38310) 2024-04-27T05:23:20.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351bdc3640 1 -- 172.21.15.138:0/4173658470 >> v1:172.21.15.114:6789/0 conn(0x562804a9b400 legacy=0x562804ba2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.765+0000 7f351bdc3640 1 -- 172.21.15.138:0/4173658470 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804a9f380 con 0x562804a9b000 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.767+0000 7f351bdc3640 1 --2- 172.21.15.138:0/4173658470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x562804ba6d50 tx=0x562804bacde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d147832b16ce5296 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.767+0000 7f351adc1640 1 -- 172.21.15.138:0/4173658470 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3ce0 con 0x562804a9b000 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.767+0000 7f351adc1640 1 -- 172.21.15.138:0/4173658470 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562804d73b00 con 0x562804a9b000 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.767+0000 7f351adc1640 1 -- 172.21.15.138:0/4173658470 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3e40 con 0x562804a9b000 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 -- 172.21.15.138:0/4173658470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 msgr2=0x562804d7c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 --2- 172.21.15.138:0/4173658470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x562804ba6d50 tx=0x562804bacde0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 -- 172.21.15.138:0/4173658470 shutdown_connections 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 --2- 172.21.15.138:0/4173658470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 -- 172.21.15.138:0/4173658470 >> 172.21.15.138:0/4173658470 conn(0x562804a9ac00 msgr2=0x562804ba2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 -- 172.21.15.138:0/4173658470 shutdown_connections 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 -- 172.21.15.138:0/4173658470 wait complete. 2024-04-27T05:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.768+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351f526880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351f526880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562804ada780 con 0x562804a9b000 2024-04-27T05:23:20.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35684/0 (socket says 172.21.15.138:35684) 2024-04-27T05:23:20.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351bdc3640 1 -- 172.21.15.138:0/3580160552 learned_addr learned my addr 172.21.15.138:0/3580160552 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351bdc3640 1 -- 172.21.15.138:0/3580160552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804a9eea0 con 0x562804a9b000 2024-04-27T05:23:20.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351bdc3640 1 --2- 172.21.15.138:0/3580160552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x56280589f560 tx=0x562804baccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeebdc0b41c9fccb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351a5c0640 1 -- 172.21.15.138:0/3580160552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab34a0 con 0x562804a9b000 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351a5c0640 1 -- 172.21.15.138:0/3580160552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562804d73e00 con 0x562804a9b000 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.769+0000 7f351a5c0640 1 -- 172.21.15.138:0/3580160552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3ce0 con 0x562804a9b000 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- 172.21.15.138:0/3580160552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 msgr2=0x562804d7c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 --2- 172.21.15.138:0/3580160552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x56280589f560 tx=0x562804baccc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- 172.21.15.138:0/3580160552 shutdown_connections 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 --2- 172.21.15.138:0/3580160552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- 172.21.15.138:0/3580160552 >> 172.21.15.138:0/3580160552 conn(0x562804a9ac00 msgr2=0x562804ba3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- 172.21.15.138:0/3580160552 shutdown_connections 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- 172.21.15.138:0/3580160552 wait complete. 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.770+0000 7f351f526880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562804ada8c0 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35688/0 (socket says 172.21.15.138:35688) 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351bdc3640 1 -- 172.21.15.138:0/2366903610 learned_addr learned my addr 172.21.15.138:0/2366903610 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351bdc3640 1 -- 172.21.15.138:0/2366903610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804a9f6c0 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351bdc3640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x5628059572c0 tx=0x562804bacc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f3518dbd640 1 -- 172.21.15.138:0/2366903610 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3e40 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f3518dbd640 1 -- 172.21.15.138:0/2366903610 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56280587a000 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562804a9f1e0 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f3518dbd640 1 -- 172.21.15.138:0/2366903610 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab34a0 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.771+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562804b0cea0 con 0x562804a9b000 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f3518dbd640 1 -- 172.21.15.138:0/2366903610 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562804ac4b00 con 0x562804a9b000 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351f526880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351f526880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562804adac80 con 0x562805892c00 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f3518dbd640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562805893c00 0x562804d7d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f3518dbd640 1 -- 172.21.15.138:0/2366903610 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562804aada40 con 0x562804a9b000 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351c5c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351c5c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35694/0 (socket says 172.21.15.138:35694) 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351c5c4640 1 -- 172.21.15.138:0/3327311199 learned_addr learned my addr 172.21.15.138:0/3327311199 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351bdc3640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562805893c00 0x562804d7d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351c5c4640 1 -- 172.21.15.138:0/3327311199 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804b0cd00 con 0x562805892c00 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351bdc3640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562805893c00 0x562804d7d080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x562805994780 tx=0x562804badc20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.772+0000 7f351c5c4640 1 --2- 172.21.15.138:0/3327311199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5628059576e0 tx=0x562804badf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1924b88b0f478e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f35165b8640 1 -- 172.21.15.138:0/3327311199 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3e40 con 0x562805892c00 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f35165b8640 1 -- 172.21.15.138:0/3327311199 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56280587a600 con 0x562805892c00 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f35165b8640 1 -- 172.21.15.138:0/3327311199 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab34a0 con 0x562805892c00 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- 172.21.15.138:0/3327311199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 msgr2=0x562804d7cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 --2- 172.21.15.138:0/3327311199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5628059576e0 tx=0x562804badf80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- 172.21.15.138:0/3327311199 shutdown_connections 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 --2- 172.21.15.138:0/3327311199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- 172.21.15.138:0/3327311199 >> 172.21.15.138:0/3327311199 conn(0x562804a9b400 msgr2=0x562804ba5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- 172.21.15.138:0/3327311199 shutdown_connections 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- 172.21.15.138:0/3327311199 wait complete. 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.773+0000 7f351f526880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562804abe280 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351c5c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351c5c4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35710/0 (socket says 172.21.15.138:35710) 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351c5c4640 1 -- 172.21.15.138:0/2211704980 learned_addr learned my addr 172.21.15.138:0/2211704980 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351c5c4640 1 -- 172.21.15.138:0/2211704980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804b0cb60 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351c5c4640 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x562805956b40 tx=0x562804badb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f3514db5640 1 -- 172.21.15.138:0/2211704980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3e40 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f3514db5640 1 -- 172.21.15.138:0/2211704980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56280587a780 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f3514db5640 1 -- 172.21.15.138:0/2211704980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab34a0 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562804b0c9c0 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.774+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562804b0c820 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f3514db5640 1 -- 172.21.15.138:0/2211704980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562804ac5600 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f3514db5640 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e18000 0x562804d7d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f3514db5640 1 -- 172.21.15.138:0/2211704980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562804aadc00 con 0x562805892c00 2024-04-27T05:23:20.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f351b5c2640 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e18000 0x562804d7d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f351f526880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f351f526880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f351b5c2640 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e18000 0x562804d7d600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x562804ba7380 tx=0x5628059f6e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.775+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351f526880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351f526880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562804abf180 con 0x562805893400 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35722/0 (socket says 172.21.15.138:35722) 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351bdc3640 1 -- 172.21.15.138:0/3177476054 learned_addr learned my addr 172.21.15.138:0/3177476054 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351bdc3640 1 -- 172.21.15.138:0/3177476054 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804b0c680 con 0x562805893400 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f351bdc3640 1 --2- 172.21.15.138:0/3177476054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x562805994900 tx=0x562804badce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f831251a643519e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f3502d91640 1 -- 172.21.15.138:0/3177476054 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3ce0 con 0x562805893400 2024-04-27T05:23:20.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f3502d91640 1 -- 172.21.15.138:0/3177476054 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56280587a300 con 0x562805893400 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.776+0000 7f3502d91640 1 -- 172.21.15.138:0/3177476054 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3e40 con 0x562805893400 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- 172.21.15.138:0/3177476054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 msgr2=0x562804d7db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 --2- 172.21.15.138:0/3177476054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x562805994900 tx=0x562804badce0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- 172.21.15.138:0/3177476054 shutdown_connections 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 --2- 172.21.15.138:0/3177476054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- 172.21.15.138:0/3177476054 >> 172.21.15.138:0/3177476054 conn(0x562805893000 msgr2=0x562804ba5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- 172.21.15.138:0/3177476054 shutdown_connections 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- 172.21.15.138:0/3177476054 wait complete. 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 Processor -- start 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- start start 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351f526880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x562804abeb40 con 0x562805893400 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351bdc3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35732/0 (socket says 172.21.15.138:35732) 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.777+0000 7f351bdc3640 1 -- 172.21.15.138:0/3231598302 learned_addr learned my addr 172.21.15.138:0/3231598302 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351bdc3640 1 -- 172.21.15.138:0/3231598302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562804b0c4e0 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351bdc3640 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x562804df85a0 tx=0x562804badbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f350158e640 1 -- 172.21.15.138:0/3231598302 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab34a0 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f350158e640 1 -- 172.21.15.138:0/3231598302 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56280587a480 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f350158e640 1 -- 172.21.15.138:0/3231598302 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562804ab3ce0 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562804a9f380 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562804a9eea0 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f34ff58a640 20 reqs_thread_entry: start 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562804e48000 0x562804d7e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562804e48400 con 0x562804e48000 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.778+0000 7f351b5c2640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562804e48000 0x562804d7e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f350158e640 1 -- 172.21.15.138:0/3231598302 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x562804ac5b80 con 0x562805893400 2024-04-27T05:23:20.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351b5c2640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562804e48000 0x562804d7e100 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f350158e640 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e49400 0x562804d7ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f350158e640 1 -- 172.21.15.138:0/3231598302 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562804aaddc0 con 0x562805893400 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351c5c4640 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e49400 0x562804d7ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351b5c2640 1 -- 172.21.15.138:0/2366903610 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562804e9c240 con 0x562804e48000 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562804e48800 con 0x562804e48000 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351c5c4640 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e49400 0x562804d7ec00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x562805a05680 tx=0x5628059f7b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351b5c2640 1 -- 172.21.15.138:0/2366903610 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562804e9c240 con 0x562804e48000 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562804e48c00 0x562804d7e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:3 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562804e49000 con 0x562804e48c00 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.779+0000 7f351bdc3640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562804e48c00 0x562804d7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.780+0000 7f351bdc3640 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562804e48c00 0x562804d7e680 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.780+0000 7f351bdc3640 1 -- 172.21.15.138:0/2366903610 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(3 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [call out=24b,read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+444 (crc 0 0 0) 0x562804e9c900 con 0x562804e48c00 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.780+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e18000 msgr2=0x562804d7d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.780+0000 7f351f526880 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e18000 0x562804d7d600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x562804ba7380 tx=0x5628059f6e40 comp rx=0 tx=0).stop 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.780+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 msgr2=0x562804d7cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.780+0000 7f351f526880 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x562805956b40 tx=0x562804badb00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 shutdown_connections 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e18000 0x562804d7d600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2211704980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805892c00 0x562804d7cb00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 >> 172.21.15.138:0/2211704980 conn(0x562804a9b400 msgr2=0x562804ba5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 shutdown_connections 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2211704980 wait complete. 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e49400 msgr2=0x562804d7ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e49400 0x562804d7ec00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x562805a05680 tx=0x5628059f7b60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 msgr2=0x562804d7db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x562804df85a0 tx=0x562804badbc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 shutdown_connections 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562804e49400 0x562804d7ec00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/3231598302 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562805893400 0x562804d7db80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 >> 172.21.15.138:0/3231598302 conn(0x562805893000 msgr2=0x562804ba5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 shutdown_connections 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/3231598302 wait complete. 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562804e48000 msgr2=0x562804d7e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562804e48000 0x562804d7e100 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562804e48c00 msgr2=0x562804d7e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562804e48c00 0x562804d7e680 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562805893c00 msgr2=0x562804d7d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562805893c00 0x562804d7d080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x562805994780 tx=0x562804badc20 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 msgr2=0x562804d7c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x5628059572c0 tx=0x562804bacc60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 shutdown_connections 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562804e48c00 0x562804d7e680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562804e48000 0x562804d7e100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562805893c00 0x562804d7d080 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 --2- 172.21.15.138:0/2366903610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562804a9b000 0x562804d7c580 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 >> 172.21.15.138:0/2366903610 conn(0x562804a9ac00 msgr2=0x562804ba3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 shutdown_connections 2024-04-27T05:23:20.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.781+0000 7f351f526880 1 -- 172.21.15.138:0/2366903610 wait complete. 2024-04-27T05:23:20.799 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "7c1a7efe-08a0-41e8-a0e4-e18cf496a33b", 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "api_name": "default", 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "is_master": true, 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "endpoints": [], 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "hostnames": [], 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "hostnames_s3website": [], 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "master_zone": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "zones": [ 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "id": "2f19b8ff-2608-4d86-8f1c-430623205476", 2024-04-27T05:23:20.800 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default", 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "endpoints": [], 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "log_meta": false, 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "log_data": false, 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_index_max_shards": 11, 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "read_only": false, 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "tier_type": "", 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "sync_from_all": true, 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "sync_from": [], 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect_zone": "", 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "supported_features": [ 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "compress-encrypted", 2024-04-27T05:23:20.801 INFO:teuthology.orchestra.run.smithi138.stdout: "resharding" 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_targets": [ 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default-placement", 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_classes": [ 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "STANDARD" 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "default-placement", 2024-04-27T05:23:20.802 INFO:teuthology.orchestra.run.smithi138.stdout: "realm_id": "", 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout: "sync_policy": { 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout: "groups": [] 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled_features": [ 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout: "resharding" 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-27T05:23:20.803 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:20.803 DEBUG:tasks.util.rgw: json result: {'id': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': '2f19b8ff-2608-4d86-8f1c-430623205476', 'zones': [{'id': '2f19b8ff-2608-4d86-8f1c-430623205476', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['compress-encrypted', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['resharding']} 2024-04-27T05:23:20.806 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-04-27T05:23:20.822 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-27T05:23:20.822 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-27T05:23:20.823 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-04-27T05:23:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setuser ceph since I am not root 2024-04-27T05:23:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T05:23:20.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.890+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.890+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.890+0000 7fdf4c813880 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562112981000 0x562112c62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.890+0000 7fdf4c813880 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562112c58600 con 0x562112981400 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.890+0000 7fdf4c813880 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x5621129c0280 con 0x562112981000 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.891+0000 7fdf48cf4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562112981000 0x562112c62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.891+0000 7fdf48cf4640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x562112981000 0x562112c62580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35734/0 (socket says 172.21.15.138:35734) 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.891+0000 7fdf48cf4640 1 -- 172.21.15.138:0/1696297783 learned_addr learned my addr 172.21.15.138:0/1696297783 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.891+0000 7fdf484f3640 1 --1- 172.21.15.138:0/1696297783 >> v1:172.21.15.114:6789/0 conn(0x562112981400 0x562112a88800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.138:38324/0 (socket says 172.21.15.138:38324) 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.891+0000 7fdf48cf4640 1 -- 172.21.15.138:0/1696297783 >> v1:172.21.15.114:6789/0 conn(0x562112981400 legacy=0x562112a88800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.891+0000 7fdf48cf4640 1 -- 172.21.15.138:0/1696297783 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562112985380 con 0x562112981000 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/1696297783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x562112a8cd50 tx=0x562112a92de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301602ca29cae103 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf47cf2640 1 -- 172.21.15.138:0/1696297783 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112999ce0 con 0x562112981000 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf47cf2640 1 -- 172.21.15.138:0/1696297783 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562112c59b00 con 0x562112981000 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf47cf2640 1 -- 172.21.15.138:0/1696297783 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112999e40 con 0x562112981000 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 -- 172.21.15.138:0/1696297783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 msgr2=0x562112c62580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 --2- 172.21.15.138:0/1696297783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x562112a8cd50 tx=0x562112a92de0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 -- 172.21.15.138:0/1696297783 shutdown_connections 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 --2- 172.21.15.138:0/1696297783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 -- 172.21.15.138:0/1696297783 >> 172.21.15.138:0/1696297783 conn(0x562112980c00 msgr2=0x562112a88000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 -- 172.21.15.138:0/1696297783 shutdown_connections 2024-04-27T05:23:20.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.893+0000 7fdf4c813880 1 -- 172.21.15.138:0/1696297783 wait complete. 2024-04-27T05:23:20.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf4c813880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf4c813880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621129c0780 con 0x562112981000 2024-04-27T05:23:20.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf48cf4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf48cf4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35740/0 (socket says 172.21.15.138:35740) 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.894+0000 7fdf48cf4640 1 -- 172.21.15.138:0/706100407 learned_addr learned my addr 172.21.15.138:0/706100407 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf48cf4640 1 -- 172.21.15.138:0/706100407 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562112984ea0 con 0x562112981000 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/706100407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x562113785560 tx=0x562112a92cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2bbd67fa935570 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf474f1640 1 -- 172.21.15.138:0/706100407 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5621129994a0 con 0x562112981000 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf474f1640 1 -- 172.21.15.138:0/706100407 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562112c59e00 con 0x562112981000 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf474f1640 1 -- 172.21.15.138:0/706100407 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112999ce0 con 0x562112981000 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 -- 172.21.15.138:0/706100407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 msgr2=0x562112c62580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 --2- 172.21.15.138:0/706100407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x562113785560 tx=0x562112a92cc0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 -- 172.21.15.138:0/706100407 shutdown_connections 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 --2- 172.21.15.138:0/706100407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 -- 172.21.15.138:0/706100407 >> 172.21.15.138:0/706100407 conn(0x562112980c00 msgr2=0x562112a89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 -- 172.21.15.138:0/706100407 shutdown_connections 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.895+0000 7fdf4c813880 1 -- 172.21.15.138:0/706100407 wait complete. 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf4c813880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf4c813880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621129c08c0 con 0x562112981000 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf48cf4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf48cf4640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35754/0 (socket says 172.21.15.138:35754) 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf48cf4640 1 -- 172.21.15.138:0/230776846 learned_addr learned my addr 172.21.15.138:0/230776846 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf48cf4640 1 -- 172.21.15.138:0/230776846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621129856c0 con 0x562112981000 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x56211383d2c0 tx=0x562112a92c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf45cee640 1 -- 172.21.15.138:0/230776846 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112999e40 con 0x562112981000 2024-04-27T05:23:20.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf45cee640 1 -- 172.21.15.138:0/230776846 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562113760000 con 0x562112981000 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5621129851e0 con 0x562112981000 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf45cee640 1 -- 172.21.15.138:0/230776846 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5621129994a0 con 0x562112981000 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.896+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621129f2ea0 con 0x562112981000 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf45cee640 1 -- 172.21.15.138:0/230776846 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5621129aab00 con 0x562112981000 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf4c813880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf4c813880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621129a4280 con 0x562113778c00 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf45cee640 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562113779c00 0x562112c63080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf45cee640 1 -- 172.21.15.138:0/230776846 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562112993a40 con 0x562112981000 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf494f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf494f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35772/0 (socket says 172.21.15.138:35772) 2024-04-27T05:23:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf494f5640 1 -- 172.21.15.138:0/3266173047 learned_addr learned my addr 172.21.15.138:0/3266173047 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.897+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562113779c00 0x562112c63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf494f5640 1 -- 172.21.15.138:0/3266173047 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621129f2d00 con 0x562113778c00 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562113779c00 0x562112c63080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x562113884180 tx=0x562112a93c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf494f5640 1 --2- 172.21.15.138:0/3266173047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56211383de60 tx=0x562112a93f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cb699017d2e656 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf434e9640 1 -- 172.21.15.138:0/3266173047 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112999e40 con 0x562113778c00 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf434e9640 1 -- 172.21.15.138:0/3266173047 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562113760600 con 0x562113778c00 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf434e9640 1 -- 172.21.15.138:0/3266173047 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5621129994a0 con 0x562113778c00 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 -- 172.21.15.138:0/3266173047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 msgr2=0x562112c62b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 --2- 172.21.15.138:0/3266173047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56211383de60 tx=0x562112a93f80 comp rx=0 tx=0).stop 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 -- 172.21.15.138:0/3266173047 shutdown_connections 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 --2- 172.21.15.138:0/3266173047 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 -- 172.21.15.138:0/3266173047 >> 172.21.15.138:0/3266173047 conn(0x562112981400 msgr2=0x562112a8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 -- 172.21.15.138:0/3266173047 shutdown_connections 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.898+0000 7fdf4c813880 1 -- 172.21.15.138:0/3266173047 wait complete. 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf4c813880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf4c813880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621129a5180 con 0x562113778c00 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf494f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf494f5640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35776/0 (socket says 172.21.15.138:35776) 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 learned_addr learned my addr 172.21.15.138:0/502363330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621129f2b60 con 0x562113778c00 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf494f5640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x56211383cb70 tx=0x562112a93b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf41ce6640 1 -- 172.21.15.138:0/502363330 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112999e40 con 0x562113778c00 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf41ce6640 1 -- 172.21.15.138:0/502363330 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562113760780 con 0x562113778c00 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.899+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5621129f29c0 con 0x562113778c00 2024-04-27T05:23:20.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf41ce6640 1 -- 172.21.15.138:0/502363330 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5621129994a0 con 0x562113778c00 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621129f2820 con 0x562113778c00 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf41ce6640 1 -- 172.21.15.138:0/502363330 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5621129ab600 con 0x562113778c00 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf41ce6640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562112cfe000 0x562112c63600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf41ce6640 1 -- 172.21.15.138:0/502363330 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562112993c00 con 0x562113778c00 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf484f3640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562112cfe000 0x562112c63600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562113779000 0x562112c63b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.900+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562113779400 con 0x562113779000 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf494f5640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562113779000 0x562112c63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf484f3640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562112cfe000 0x562112c63600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x562112a8d380 tx=0x5621138d0f60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf494f5640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562113779000 0x562112c63b80 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 realm 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562113779800 con 0x562113779000 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cff000 con 0x562113779000 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.901+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cff400 con 0x562113779000 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562112cff800 0x562112c64100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:5 5.11 5:8d97b3da:::zone_info.2f19b8ff-2608-4d86-8f1c-430623205476:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cffc00 con 0x562112cff800 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562112cff800 0x562112c64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562112cff800 0x562112c64100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 1 ==== osd_op_reply(5 zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 [read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 20 rados_obj.operate() r=0 bl.length=874 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 20 searching for the correct realm 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.902+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cff000 con 0x562112cff800 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf48cf4640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562112cff000 0x562112c64c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cffc00 con 0x562112cff000 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf484f3640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562112cff000 0x562112c64c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf484f3640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562112cff000 0x562112c64c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf484f3640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562112cffc00 0x562112c65180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.903+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.904+0000 7fdf494f5640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562112cffc00 0x562112c65180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.904+0000 7fdf494f5640 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562112cffc00 0x562112c65180 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.904+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.904+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.904+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.904+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff000 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.905+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff000 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.906+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=107b] v31'4 uv4 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.907+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v31'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.908+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.909+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff000 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff800 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cffc00 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562113779000 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v31'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.910+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cfec00 con 0x562112cff000 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 RGWRados::pool_iterate: got zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 RGWRados::pool_iterate: got zone_info.2f19b8ff-2608-4d86-8f1c-430623205476 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cff400 con 0x562113779000 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562113779400 con 0x562113779000 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:40 5.2 5:4023c2a0:::zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562113779800 con 0x562112cff800 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 11 ==== osd_op_reply(40 zonegroup_info.7c1a7efe-08a0-41e8-a0e4-e18cf496a33b [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados_obj.operate() r=0 bl.length=420 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 zone default found 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 4 Realm: () 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 4 ZoneGroup: default (7c1a7efe-08a0-41e8-a0e4-e18cf496a33b) 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 4 Zone: default (2f19b8ff-2608-4d86-8f1c-430623205476) 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 zonegroup default 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.911+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28000 con 0x562112cffc00 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28400 con 0x562112cff800 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28800 con 0x562112cff800 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 started sync module instance, tier type = 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 20 started zone id=2f19b8ff-2608-4d86-8f1c-430623205476 (name=default) with tier type = 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.912+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28c00 con 0x562112cff000 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.913+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 7 ==== osd_op_reply(44 notify.0 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.913+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94700048853248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29000 con 0x562112cff000 2024-04-27T05:23:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.913+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29400 con 0x562113779000 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.914+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94700048853248] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.915+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 15 ==== osd_op_reply(46 notify.1 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.915+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94700048854400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29800 con 0x562113779000 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.915+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29c00 con 0x562112cff800 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.916+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94700048854400] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.916+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 14 ==== osd_op_reply(48 notify.2 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.916+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94700048855552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28400 con 0x562112cff800 2024-04-27T05:23:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.916+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28800 con 0x562112cff000 2024-04-27T05:23:20.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.917+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 9 ==== osd_op_reply(50 notify.3 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.917+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94700048856704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29c00 con 0x562112cff000 2024-04-27T05:23:20.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.917+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28000 con 0x562113779000 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.918+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94700048855552] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.918+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94700048856704] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.918+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 17 ==== osd_op_reply(52 notify.4 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.918+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94700048857856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562113779800 con 0x562113779000 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.918+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562113779400 con 0x562112cffc00 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.919+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94700048857856] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.919+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 11 ==== osd_op_reply(54 notify.5 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.919+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94700048859008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112cff400 con 0x562112cffc00 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.919+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5621138e5800 con 0x562112cffc00 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.920+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94700048859008] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.920+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 13 ==== osd_op_reply(56 notify.6 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.920+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94700048860160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5621138e5400 con 0x562112cffc00 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.920+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28c00 con 0x562112cffc00 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.921+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94700048860160] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 15 ==== osd_op_reply(58 notify.7 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94700048861312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29000 con 0x562112cffc00 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=0 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=1 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=2 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=3 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=4 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=5 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.923+0000 7fdf4c813880 20 add_watcher() i=6 2024-04-27T05:23:20.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.924+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94700048861312] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.924+0000 7fdf4c813880 20 add_watcher() i=7 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.924+0000 7fdf4c813880 2 all 8 watchers are set, enabling cache 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.924+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d28800 con 0x562113779000 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.924+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.924+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94700048862464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112d29400 con 0x562113779000 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.925+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94700048862464] v31'245 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.925+0000 7fdf2fcc2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.925+0000 7fdf4c813880 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.925+0000 7fdf4c813880 20 check_secure_mon_conn(): mode 1 is insecure 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf4c813880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf4c813880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621129a4780 con 0x562112cfec00 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf484f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf484f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35788/0 (socket says 172.21.15.138:35788) 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf484f3640 1 -- 172.21.15.138:0/526445513 learned_addr learned my addr 172.21.15.138:0/526445513 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf484f3640 1 -- 172.21.15.138:0/526445513 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621129f2680 con 0x562112cfec00 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.926+0000 7fdf484f3640 1 --2- 172.21.15.138:0/526445513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x562112ce0ae0 tx=0x5621138d0f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b609467eddc425a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf2f4c1640 1 -- 172.21.15.138:0/526445513 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112d2a580 con 0x562112cfec00 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf2f4c1640 1 -- 172.21.15.138:0/526445513 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562113760000 con 0x562112cfec00 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf2f4c1640 1 -- 172.21.15.138:0/526445513 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112d2a6e0 con 0x562112cfec00 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 -- 172.21.15.138:0/526445513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 msgr2=0x562112c64680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 --2- 172.21.15.138:0/526445513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x562112ce0ae0 tx=0x5621138d0f00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 -- 172.21.15.138:0/526445513 shutdown_connections 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 --2- 172.21.15.138:0/526445513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 -- 172.21.15.138:0/526445513 >> 172.21.15.138:0/526445513 conn(0x562112d29800 msgr2=0x562112a8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 -- 172.21.15.138:0/526445513 shutdown_connections 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.927+0000 7fdf4c813880 1 -- 172.21.15.138:0/526445513 wait complete. 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf4c813880 1 Processor -- start 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf4c813880 1 -- start start 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf4c813880 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf4c813880 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621129a4500 con 0x562112cfec00 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf484f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf484f3640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.138:35802/0 (socket says 172.21.15.138:35802) 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf484f3640 1 -- 172.21.15.138:0/4106936500 learned_addr learned my addr 172.21.15.138:0/4106936500 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf484f3640 1 -- 172.21.15.138:0/4106936500 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621129f24e0 con 0x562112cfec00 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf484f3640 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x562112ce1860 tx=0x5621138d1080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.928+0000 7fdf2dcbe640 1 -- 172.21.15.138:0/4106936500 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112d2a840 con 0x562112cfec00 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.929+0000 7fdf2dcbe640 1 -- 172.21.15.138:0/4106936500 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562113760a80 con 0x562112cfec00 2024-04-27T05:23:20.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.929+0000 7fdf2dcbe640 1 -- 172.21.15.138:0/4106936500 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562112d2a580 con 0x562112cfec00 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.929+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562112985380 con 0x562112cfec00 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.929+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562112984ea0 con 0x562112cfec00 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.929+0000 7fdf2dcbe640 1 -- 172.21.15.138:0/4106936500 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x5621129abb80 con 0x562112cfec00 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.929+0000 7fdf2dcbe640 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5621138e5800 0x562112c65700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf2dcbe640 1 -- 172.21.15.138:0/4106936500 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x562112993dc0 con 0x562112cfec00 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf4c813880 5 note: GC not initialized 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf494f5640 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5621138e5800 0x562112c65700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf2b4b9640 20 reqs_thread_entry: start 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf4c813880 20 init_complete bucket index max shards: 11 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf4c813880 10 rgw notify: Started notification manager with: 1 workers 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf494f5640 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5621138e5800 0x562112c65700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x562113882ff0 tx=0x5621138d1ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf294b5640 20 rgw notify: INFO: next queues processing will happen at: Sat Apr 27 05:23:51 2024 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-27T05:23:20.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf28c93640 20 reqs_thread_entry: start 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf4c813880 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.930+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df8000 con 0x562112cffc00 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 17 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 20 rados->read ofs=0 len=0 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df8400 con 0x562113779000 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 21 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-04-27T05:23:20.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.931+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~439 in=439b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df8800 con 0x562112cffc00 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 18 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~439] v31'6 uv6 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf4c813880 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf4c813880 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf4c813880 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94700048864768 in=627b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df8c00 con 0x562113779000 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 22 ==== watch-notify(notify (1) cookie 94700048857856 notify 133143986197 ret 0) v3 ==== 657+0+0 (crc 0 0 0) 0x562112cc7d40 con 0x562113779000 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf434e9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986197 cookie 94700048857856 notifier 4917 bl.length()=615 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 23 ==== osd_op_reply(65 notify.4 [notify cookie 94700048864768 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf434e9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.932+0000 7fdf434e9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.933+0000 7fdf434e9640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0a000 con 0x562113779000 2024-04-27T05:23:20.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.933+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 24 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.933+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 25 ==== watch-notify(notify_complete (2) cookie 94700048864768 notify 133143986197 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562112d81a00 con 0x562113779000 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.933+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df9000 con 0x562113779000 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 26 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v31'21 uv21 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf4c813880 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf4c813880 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf4c813880 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94700048864768 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df9400 con 0x562112cff800 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 16 ==== watch-notify(notify (1) cookie 94700048855552 notify 133143986199 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x562113880d00 con 0x562112cff800 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 17 ==== osd_op_reply(68 notify.2 [notify cookie 94700048864768 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf434e9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986199 cookie 94700048855552 notifier 4917 bl.length()=177 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf434e9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf434e9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T05:23:20.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.934+0000 7fdf434e9640 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0a400 con 0x562112cff800 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.935+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 18 ==== watch-notify(notify_complete (2) cookie 94700048864768 notify 133143986199 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562113880ea0 con 0x562112cff800 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.935+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 19 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "user_id": "ada", 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "display_name": "Ms. Admin User", 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "email": "", 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "suspended": 0, 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "max_buckets": 0, 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "subusers": [], 2024-04-27T05:23:20.955 INFO:teuthology.orchestra.run.smithi138.stdout: "keys": [ 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "user": "ada", 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "access_key": "MH1WC2XQ1S8UISFDZC8W", 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "secret_key": "dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG" 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "swift_keys": [], 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": [ 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "buckets", 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "*" 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "info", 2024-04-27T05:23:20.956 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "read" 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "ratelimit", 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "*" 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "usage", 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "*" 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "users", 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "*" 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "zone", 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "perm": "*" 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "op_mask": "read, write, delete", 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "default_placement": "", 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "default_storage_class": "", 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "placement_tags": [], 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "bucket_quota": { 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:23:20.958 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "user_quota": { 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": false, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "check_on_raw": false, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size": -1, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "max_size_kb": 0, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "max_objects": -1 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "temp_url_keys": [], 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "rgw", 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: "mfa_ids": [] 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-27T05:23:20.959 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.935+0000 7fdf2fcc2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.935+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94700048862464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df9800 con 0x562113779000 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.937+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94700048862464] v31'246 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.937+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94700048853248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112df9c00 con 0x562112cff000 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.938+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94700048853248] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.938+0000 7fdf4c813880 20 remove_watcher() i=0 2024-04-27T05:23:20.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.938+0000 7fdf4c813880 2 removed watcher, disabling cache 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.938+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94700048854400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0a000 con 0x562113779000 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.939+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94700048854400] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.939+0000 7fdf4c813880 20 remove_watcher() i=1 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.939+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94700048855552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0a800 con 0x562112cff800 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.940+0000 7fdf48cf4640 1 -- 172.21.15.138:0/502363330 <== osd.3 v2:172.21.15.138:6808/29866 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94700048855552] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1afc0 con 0x562112cff800 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.940+0000 7fdf4c813880 20 remove_watcher() i=2 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.940+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94700048856704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0ac00 con 0x562112cff000 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.941+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 <== osd.2 v2:172.21.15.138:6809/29865 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94700048856704] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1b200 con 0x562112cff000 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.941+0000 7fdf4c813880 20 remove_watcher() i=3 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.941+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94700048857856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0b000 con 0x562113779000 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.942+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.1 v2:172.21.15.138:6800/29864 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94700048857856] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562113779000 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.942+0000 7fdf4c813880 20 remove_watcher() i=4 2024-04-27T05:23:20.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.942+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94700048859008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0b400 con 0x562112cffc00 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.943+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94700048859008] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.943+0000 7fdf4c813880 20 remove_watcher() i=5 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.943+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94700048860160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0b800 con 0x562112cffc00 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.944+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94700048860160] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.944+0000 7fdf4c813880 20 remove_watcher() i=6 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.944+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 --> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94700048861312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562112e0bc00 con 0x562112cffc00 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf494f5640 1 -- 172.21.15.138:0/502363330 <== osd.0 v2:172.21.15.114:6800/27747 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94700048861312] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562112d1a900 con 0x562112cffc00 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 20 remove_watcher() i=7 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562112cffc00 msgr2=0x562112c65180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:6800/27747,v1:172.21.15.114:6801/27747] conn(0x562112cffc00 0x562112c65180 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562113779000 msgr2=0x562112c63b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6800/29864,v1:172.21.15.138:6801/29864] conn(0x562113779000 0x562112c63b80 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562112cff000 msgr2=0x562112c64c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6809/29865,v1:172.21.15.138:6811/29865] conn(0x562112cff000 0x562112c64c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562112cff800 msgr2=0x562112c64100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6808/29866,v1:172.21.15.138:6810/29866] conn(0x562112cff800 0x562112c64100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562112cfe000 msgr2=0x562112c63600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562112cfe000 0x562112c63600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x562112a8d380 tx=0x5621138d0f60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 msgr2=0x562112c62b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 --2- 172.21.15.138:0/502363330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562113778c00 0x562112c62b00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x56211383cb70 tx=0x562112a93b00 comp rx=0 tx=0).stop 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 reap_dead start 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf484f3640 1 -- 172.21.15.138:0/502363330 reap_dead start 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.945+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 shutdown_connections 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 >> 172.21.15.138:0/502363330 conn(0x562112981400 msgr2=0x562112a8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 shutdown_connections 2024-04-27T05:23:20.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/502363330 wait complete. 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5621138e5800 msgr2=0x562112c65700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5621138e5800 0x562112c65700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x562113882ff0 tx=0x5621138d1ec0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 msgr2=0x562112c64680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x562112ce1860 tx=0x5621138d1080 comp rx=0 tx=0).stop 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 shutdown_connections 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x5621138e5800 0x562112c65700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/4106936500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112cfec00 0x562112c64680 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 >> 172.21.15.138:0/4106936500 conn(0x562112d29800 msgr2=0x562112a8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 shutdown_connections 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/4106936500 wait complete. 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562113779c00 msgr2=0x562112c63080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562113779c00 0x562112c63080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x562113884180 tx=0x562112a93c20 comp rx=0 tx=0).stop 2024-04-27T05:23:20.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 msgr2=0x562112c62580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x56211383d2c0 tx=0x562112a92c60 comp rx=0 tx=0).stop 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 shutdown_connections 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x562113779c00 0x562112c63080 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 --2- 172.21.15.138:0/230776846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562112981000 0x562112c62580 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 >> 172.21.15.138:0/230776846 conn(0x562112980c00 msgr2=0x562112a89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 shutdown_connections 2024-04-27T05:23:20.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-27T05:23:20.946+0000 7fdf4c813880 1 -- 172.21.15.138:0/230776846 wait complete. 2024-04-27T05:23:20.970 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:20.970 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:20.970 ERROR:root:0 2024-04-27T05:23:20.971 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:20.978 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx000003d0436a24b2db45b-00662c8bc8-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:23:20.978 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-04-27T05:23:20.988 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:20.989 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-04-27T05:23:21.000 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-04-27T05:23:21.001 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-04-27T05:23:21.012 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-27T05:23:21.012 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-27T05:23:21.018 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-04-27T05:23:21.019 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.022 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.022 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-04-27T05:23:21.026 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.027 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-27T05:23:21.032 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.032 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.036 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.036 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-27T05:23:21.041 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.041 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.045 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.045 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-04-27T05:23:21.053 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}] 2024-04-27T05:23:21.054 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.057 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.058 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-04-27T05:23:21.064 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.065 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.068 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.069 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-04-27T05:23:21.076 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-04-27T05:23:21.076 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.080 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.080 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-04-27T05:23:21.087 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-04-27T05:23:21.087 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.091 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.092 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-04-27T05:23:21.098 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.098 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.102 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.102 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-04-27T05:23:21.107 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.108 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.111 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.111 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-04-27T05:23:21.118 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.118 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.121 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.122 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.126 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T05:23:21.127 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-27T05:23:21.130 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T05:23:21.146 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-27T05:23:21.150 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-04-27T05:23:21.150 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T05:23:21.155 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:23:21.141273Z', 'creation_time': '2024-04-27T05:23:21.134359Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T05:23:21.155 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-27T05:23:21.160 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:23:21.141273Z', 'creation_time': '2024-04-27T05:23:21.134359Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-27T05:23:21.164 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T05:23:21.169 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:23:21.141273Z', 'creation_time': '2024-04-27T05:23:21.134359Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T05:23:21.169 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-04-27T05:23:21.173 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx00000e2bd59e66fc06931-00662c8bc9-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:23:21.178 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-04-27T05:23:21.184 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.184 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-04-27T05:23:21.193 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:23:21.200 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9'} 2024-04-27T05:23:21.210 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.259 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9'} 2024-04-27T05:23:21.269 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.269 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-04-27T05:23:21.279 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.285 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-04-27T05:23:21.292 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:23:21.292 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T05:23:21.296 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '7c1a7efe-08a0-41e8-a0e4-e18cf496a33b', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'marker': '2f19b8ff-2608-4d86-8f1c-430623205476.4231.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T05:23:21.263451Z', 'creation_time': '2024-04-27T05:23:21.134359Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T05:24:21.344 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-04-27T05:24:21.364 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'anonymous', 'categories': []}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'bar', 'categories': []}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'bud', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'fud', 'categories': []}, {'bucket': 'myfoo7', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'fud', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}]} 2024-04-27T05:24:21.364 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-04-27T05:24:21.379 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}]}, {'bucket': 'myfoo7', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4}}]} 2024-04-27T05:24:21.380 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-27T05:24:21.385 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}]} 2024-04-27T05:24:21.386 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-04-27T05:24:21.391 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}}]} 2024-04-27T05:24:21.391 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-04-27T05:24:21.395 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T05:24:21.396 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-04-27T05:24:21.400 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}]} 2024-04-27T05:24:21.400 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-04-27T05:24:21.405 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': []}, {'bucket': 'useless-bucket', 'time': '2024-04-27T05:00:00.000000Z', 'epoch': 1714194000, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}]} 2024-04-27T05:24:21.405 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-04-27T05:24:21.411 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:24:21.412 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-27T05:24:21.416 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-04-27T05:24:21.416 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-27T05:24:21.424 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:24:21.428 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-27T05:24:21.436 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:24:21.679 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-27T05:24:21.684 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000e3aa7c330cd9bb67-00662c8c05-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:21.786 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-27T05:24:21.791 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-27T05:24:21.795 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-27T05:24:21.799 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'id': '', 'email': '', 'permission': {'flags': 1}, 'name': '', 'group': 1, 'url_spec': ''}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-27T05:24:21.857 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-04-27T05:24:21.893 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:24:21.894 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-27T05:24:21.899 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-04-27T05:24:21.900 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-27T05:24:21.905 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T05:24:21.917 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-27T05:24:21.921 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000b502df75cb30c0f8-00662c8c05-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:21.971 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-04-27T05:24:22.006 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:24:22.006 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T05:24:22.010 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx000004ae4dbe7c2b05ad5-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.010 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-04-27T05:24:22.014 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': '5659e27f-2999-4d49-8a80-9d85e921a02f'}]}} 2024-04-27T05:24:22.014 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-04-27T05:24:22.025 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T05:24:22.025 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-27T05:24:22.029 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T05:24:22.029 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-04-27T05:24:22.033 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000a0f7636d9f667873-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.033 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-04-27T05:24:22.037 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx0000071990beea75a3b34-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.054 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-27T05:24:22.057 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T05:24:22.058 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-04-27T05:24:22.061 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000f69d9762ee093ab7-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.062 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-04-27T05:24:22.065 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000c655c3a237e9cec1-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.065 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-04-27T05:24:22.070 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T05:24:22.070 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-04-27T05:24:22.073 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000d868da15b5ebbad4-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.074 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-04-27T05:24:22.077 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000001d7f981cd1b92f4a-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.077 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-04-27T05:24:22.082 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000008d646f5344020390-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.082 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-04-27T05:24:22.086 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000004dc61d72dc57b29d-00662c8c06-1085-default', 'HostId': '1085-default-default'} 2024-04-27T05:24:22.086 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-04-27T05:24:22.095 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:24:22.095 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-27T05:24:22.099 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-04-27T05:24:22.099 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-04-27T05:24:22.104 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:24:22.104 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-27T05:24:22.108 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-04-27T05:24:22.108 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-04-27T05:24:22.114 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T05:24:22.118 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-27T05:24:22.127 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-04-27T05:24:22.127 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:22.128 INFO:tasks.rgw.client.0.smithi138.stdout:2024-04-27T05:24:22.127+0000 7f3029ab3640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 34390) UID: 0 2024-04-27T05:24:22.128 INFO:tasks.rgw.client.0.smithi138.stdout:2024-04-27T05:24:22.127+0000 7f302d312880 -1 shutting down 2024-04-27T05:24:28.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-04-27T05:24:28.228 INFO:tasks.rgw.client.0:Stopped 2024-04-27T05:24:28.229 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-04-27T05:24:28.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/vault-root-token 2024-04-27T05:24:28.314 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-27T05:24:28.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.428+0000 7f4541296640 1 Processor -- start 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.428+0000 7f4541296640 1 -- start start 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.428+0000 7f4541296640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f453c095b90 0x7f453c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.428+0000 7f4541296640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c06dcb0 con 0x7f453c06cf90 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.428+0000 7f4541296640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453c06ab90 con 0x7f453c095b90 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.429+0000 7f453affd640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f453c095b90 0x7f453c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.429+0000 7f4533fff640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f453c06cf90 0x7f453c06d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51260/0 (socket says 172.21.15.114:51260) 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.429+0000 7f4533fff640 1 -- 172.21.15.114:0/973745646 learned_addr learned my addr 172.21.15.114:0/973745646 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-04-27T05:24:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.429+0000 7f453affd640 1 -- 172.21.15.114:0/973745646 >> v1:172.21.15.114:6789/0 conn(0x7f453c06cf90 legacy=0x7f453c06d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:24:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.429+0000 7f453affd640 1 -- 172.21.15.114:0/973745646 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c06e7d0 con 0x7f453c095b90 2024-04-27T05:24:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.429+0000 7f453a7fc640 1 -- 172.21.15.114:0/973745646 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581981707 0 0) 0x7f453c06dcb0 con 0x7f453c06cf90 2024-04-27T05:24:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f453affd640 1 --2- 172.21.15.114:0/973745646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c095b90 0x7f453c06ca50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f452c009720 tx=0x7f452c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8284cce5af3efc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:24:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f453a7fc640 1 -- 172.21.15.114:0/973745646 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f452c02e070 con 0x7f453c095b90 2024-04-27T05:24:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f453a7fc640 1 -- 172.21.15.114:0/973745646 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c0316d0 con 0x7f453c095b90 2024-04-27T05:24:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f453a7fc640 1 -- 172.21.15.114:0/973745646 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f452c031990 con 0x7f453c095b90 2024-04-27T05:24:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 -- 172.21.15.114:0/973745646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c095b90 msgr2=0x7f453c06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 --2- 172.21.15.114:0/973745646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c095b90 0x7f453c06ca50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f452c009720 tx=0x7f452c02b840 comp rx=0 tx=0).stop 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 -- 172.21.15.114:0/973745646 shutdown_connections 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 --2- 172.21.15.114:0/973745646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c095b90 0x7f453c06ca50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 -- 172.21.15.114:0/973745646 >> 172.21.15.114:0/973745646 conn(0x7f453c098910 msgr2=0x7f453c098d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 -- 172.21.15.114:0/973745646 shutdown_connections 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.430+0000 7f4541296640 1 -- 172.21.15.114:0/973745646 wait complete. 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.431+0000 7f4541296640 1 Processor -- start 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.431+0000 7f4541296640 1 -- start start 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.431+0000 7f4541296640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 0x7f453c1be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.431+0000 7f4541296640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c1a7150 con 0x7f453c06cf90 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f453affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 0x7f453c1be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f453affd640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 0x7f453c1be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42506/0 (socket says 172.21.15.114:42506) 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f453affd640 1 -- 172.21.15.114:0/1494677854 learned_addr learned my addr 172.21.15.114:0/1494677854 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-27T05:24:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f453affd640 1 -- 172.21.15.114:0/1494677854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c009400 con 0x7f453c06cf90 2024-04-27T05:24:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f453affd640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 0x7f453c1be0d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f452c009840 tx=0x7f452c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:24:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f452c0328b0 con 0x7f453c06cf90 2024-04-27T05:24:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c032a10 con 0x7f453c06cf90 2024-04-27T05:24:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c1a92f0 con 0x7f453c06cf90 2024-04-27T05:24:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f452c032cf0 con 0x7f453c06cf90 2024-04-27T05:24:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.432+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c1a8ae0 con 0x7f453c06cf90 2024-04-27T05:24:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.433+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) v1 ==== 44817+0+0 (secure 0 0 0) 0x7f452c04d3f0 con 0x7f453c06cf90 2024-04-27T05:24:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.433+0000 7f45337fe640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f451c037700 0x7f451c039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T05:24:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.433+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c04e190 con 0x7f453c06cf90 2024-04-27T05:24:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.433+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f452c06c940 con 0x7f453c06cf90 2024-04-27T05:24:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.434+0000 7f4533fff640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f451c037700 0x7f451c039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T05:24:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.434+0000 7f4533fff640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f451c037700 0x7f451c039bc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4524000c00 tx=0x7f4524007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T05:24:28.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.438+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182227 (secure 0 0 0) 0x7f452c0306b0 con 0x7f453c06cf90 2024-04-27T05:24:28.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.607+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f453c06cd10 con 0x7f453c06cf90 2024-04-27T05:24:28.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.609+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f452c0306b0 con 0x7f453c06cf90 2024-04-27T05:24:28.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.610+0000 7f45337fe640 1 -- 172.21.15.114:0/1494677854 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f452c0302f0 con 0x7f453c06cf90 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f451c037700 msgr2=0x7f451c039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f451c037700 0x7f451c039bc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4524000c00 tx=0x7f4524007910 comp rx=0 tx=0).stop 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 msgr2=0x7f453c1be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 0x7f453c1be0d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f452c009840 tx=0x7f452c0046c0 comp rx=0 tx=0).stop 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 shutdown_connections 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.138:6824/29568,v1:172.21.15.138:6825/29568] conn(0x7f451c037700 0x7f451c039bc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 --2- 172.21.15.114:0/1494677854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f453c06cf90 0x7f453c1be0d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 >> 172.21.15.114:0/1494677854 conn(0x7f453c098910 msgr2=0x7f453c097b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 shutdown_connections 2024-04-27T05:24:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-27T05:24:28.612+0000 7f4541296640 1 -- 172.21.15.114:0/1494677854 wait complete. 2024-04-27T05:24:28.624 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-27T05:24:28.624 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-27T05:24:28.624 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-27T05:24:28.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:28.668 INFO:tasks.ceph.osd.0:Stopped 2024-04-27T05:24:28.668 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-27T05:24:28.668 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:28.699 INFO:tasks.ceph.osd.1:Stopped 2024-04-27T05:24:28.699 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-27T05:24:28.700 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:28.770 INFO:tasks.ceph.osd.2:Stopped 2024-04-27T05:24:28.770 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-27T05:24:28.771 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:28.841 INFO:tasks.ceph.osd.3:Stopped 2024-04-27T05:24:28.841 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-27T05:24:28.842 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-27T05:24:28.842 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:28.911 INFO:tasks.ceph.mgr.x:Stopped 2024-04-27T05:24:28.911 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-27T05:24:28.912 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-27T05:24:28.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T05:24:28.922 INFO:tasks.ceph.mon.a:Stopped 2024-04-27T05:24:28.923 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-27T05:24:28.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-04-27T05:24:28.989 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi114.front.sepia.ceph.com 2024-04-27T05:24:28.990 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-27T05:24:29.140 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:24:29.140 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-27T05:24:29.390 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:24:29.390 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-27T05:24:29.447 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:24:29.447 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-27T05:24:29.496 INFO:tasks.ceph:Archiving mon data... 2024-04-27T05:24:29.497 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/data/mon.a.tgz 2024-04-27T05:24:29.497 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-04-27T05:24:29.512 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.AnLNzNV8FZ 2024-04-27T05:24:29.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AnLNzNV8FZ 2024-04-27T05:24:29.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.AnLNzNV8FZ 2024-04-27T05:24:29.713 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.AnLNzNV8FZ is 216KB 2024-04-27T05:24:29.806 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.AnLNzNV8FZ 2024-04-27T05:24:29.820 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-27T05:24:29.821 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T05:24:29.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T05:24:29.891 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-27T05:24:29.891 DEBUG:teuthology.orchestra.run.smithi114:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-27T05:24:29.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-27T05:24:29.950 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-27T05:24:29.961 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-27T05:24:29.966 INFO:tasks.ceph:Archiving crash dumps... 2024-04-27T05:24:29.968 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/remote/smithi114/crash 2024-04-27T05:24:29.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T05:24:30.018 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/remote/smithi138/crash 2024-04-27T05:24:30.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T05:24:30.076 INFO:tasks.ceph:Compressing logs... 2024-04-27T05:24:30.076 DEBUG:teuthology.orchestra.run.smithi114:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T05:24:30.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T05:24:32.485 INFO:tasks.ceph:Archiving logs... 2024-04-27T05:24:32.485 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/remote/smithi114/log 2024-04-27T05:24:32.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T05:24:32.644 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/remote/smithi138/log 2024-04-27T05:24:32.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T05:24:33.020 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T05:24:33.032 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T05:24:33.032 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T05:24:33.035 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T05:24:33.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-04-27T05:24:33.102 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T05:24:33.111 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T05:24:33.111 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T05:24:33.114 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T05:24:33.125 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-04-27T05:24:33.127 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-27T05:24:33.130 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T05:24:33.130 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-04-27T05:24:33.130 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 45 +291us[ +305us] +/- 62ms 2024-04-27T05:24:33.130 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -1586us[-1572us] +/- 58ms 2024-04-27T05:24:33.130 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +1684us[+1684us] +/- 72ms 2024-04-27T05:24:33.130 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:24:33.131 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T05:24:33.131 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-27T05:24:33.132 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 43 +295us[ +295us] +/- 62ms 2024-04-27T05:24:33.132 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -1579us[-1568us] +/- 58ms 2024-04-27T05:24:33.132 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +1697us[+1697us] +/- 72ms 2024-04-27T05:24:33.132 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T05:24:33.132 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T05:24:33.143 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T05:24:33.144 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T05:24:33.153 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T05:24:33.228 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T05:24:33.285 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T05:24:33.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:24:33.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-27T05:24:33.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:24:33.337 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T05:24:33.347 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T05:24:33.357 INFO:teuthology.task.internal:Duration was 593.165123 seconds 2024-04-27T05:24:33.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T05:24:33.368 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T05:24:33.368 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T05:24:33.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T05:24:33.404 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T05:24:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T05:24:33.833 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T05:24:33.833 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-04-27T05:24:33.834 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T05:24:33.893 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-27T05:24:33.893 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T05:24:33.917 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T05:24:33.917 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T05:24:33.936 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T05:24:33.982 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T05:24:33.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T05:24:34.001 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T05:24:34.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T05:24:34.124 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T05:24:34.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T05:24:34.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T05:24:34.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T05:24:34.181 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T05:24:34.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T05:24:34.209 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-04-27T05:24:34.233 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-27T05:24:34.251 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T05:24:34.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:24:34.285 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T05:24:34.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T05:24:34.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T05:24:34.315 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T05:24:34.316 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/remote/smithi114 2024-04-27T05:24:34.316 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T05:24:34.358 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_22:40:03-rgw-reef-distro-default-smithi/7675743/remote/smithi138 2024-04-27T05:24:34.358 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T05:24:34.393 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T05:24:34.394 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T05:24:34.396 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T05:24:34.443 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T05:24:34.454 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T05:24:34.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T05:24:34.500 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T05:24:34.500 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T05:24:34.502 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T05:24:34.514 INFO:teuthology.orchestra.run.smithi114.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 05:24 /home/ubuntu/cephtest 2024-04-27T05:24:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 05:24 /home/ubuntu/cephtest 2024-04-27T05:24:34.518 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T05:24:34.529 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T05:24:34.553 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} duration: 593.1651225090027 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-27T05:24:34.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T05:24:34.637 INFO:teuthology.run:pass